[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:09:21 dist-test-slave-drl0 krb5kdc[17596](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:09:21 dist-test-slave-drl0 krb5kdc[17596](info): set up 2 sockets
Aug 12 11:09:21 dist-test-slave-drl0 krb5kdc[17596](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 11:09:23.073640 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.300s user 0.006s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:09:23 dist-test-slave-drl0 krb5kdc[17596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:09:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:09:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:09:23.214414 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:40401
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:46575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:40401
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:23.359236 17616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:23.359509 17616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:23.359550 17616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:23.364283 17616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:09:23.364346 17616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:23.364367 17616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:23.364383 17616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:09:23.364400 17616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:09:23.370594 17616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:46575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:40401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:40401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:23.372008 17616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:23.373339 17616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:23.385170 17631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:23.385241 17626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:23.386019 17616 server_base.cc:1034] running on GCE node
W20260812 11:09:23.386468 17624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:23.387394 17616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:23.389367 17616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:23.390565 17616 hybrid_clock.cc:648] HybridClock initialized: now 1786532963390471 us; error 112 us; skew 500 ppm
Aug 12 11:09:23 dist-test-slave-drl0 krb5kdc[17596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532963, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:23.394534 17616 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:09:23.395646 17616 webserver.cc:460] Webserver started at http://127.17.44.254:33885/ using document root <none> and password file <none>
I20260812 11:09:23.396337 17616 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:23.396406 17616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:23.396673 17616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:23.399001 17616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "28e851a392b94a6a8f9cb9df3437d7ee"
format_stamp: "Formatted at 2026-08-12 11:09:23 on dist-test-slave-drl0"
server_key: "6d2fb1162d63583594bd8a4eec08ca9e"
server_key_iv: "d94751876e2684dad37ef2dd48b3a379"
server_key_version: "encryptionkey@0"
I20260812 11:09:23.399605 17616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "28e851a392b94a6a8f9cb9df3437d7ee"
format_stamp: "Formatted at 2026-08-12 11:09:23 on dist-test-slave-drl0"
server_key: "6d2fb1162d63583594bd8a4eec08ca9e"
server_key_iv: "d94751876e2684dad37ef2dd48b3a379"
server_key_version: "encryptionkey@0"
I20260812 11:09:23.404837 17616 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 11:09:23.407713 17640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:23.409157 17616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:09:23.409315 17616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "28e851a392b94a6a8f9cb9df3437d7ee"
format_stamp: "Formatted at 2026-08-12 11:09:23 on dist-test-slave-drl0"
server_key: "6d2fb1162d63583594bd8a4eec08ca9e"
server_key_iv: "d94751876e2684dad37ef2dd48b3a379"
server_key_version: "encryptionkey@0"
I20260812 11:09:23.409439 17616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:23.443378 17616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:23.448191 17616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:23.448513 17616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:23.465626 17616 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:40401
I20260812 11:09:23.465811 17735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:40401 every 8 connection(s)
I20260812 11:09:23.466995 17616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:09:23.471242 17736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:23.473692 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 17616
I20260812 11:09:23.473881 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:09:23.475904 17587 external_mini_cluster.cc:1428] Setting key 47059b3c0749721fbe97a064c622e0b4
I20260812 11:09:23.481155 17736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28e851a392b94a6a8f9cb9df3437d7ee: Bootstrap starting.
I20260812 11:09:23.484853 17736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28e851a392b94a6a8f9cb9df3437d7ee: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:23.486305 17736 log.cc:824] T 00000000000000000000000000000000 P 28e851a392b94a6a8f9cb9df3437d7ee: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:23.491469 17736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28e851a392b94a6a8f9cb9df3437d7ee: No bootstrap required, opened a new log
I20260812 11:09:23.498631 17736 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28e851a392b94a6a8f9cb9df3437d7ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28e851a392b94a6a8f9cb9df3437d7ee" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 40401 } }
I20260812 11:09:23.498970 17736 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28e851a392b94a6a8f9cb9df3437d7ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:23.499084 17736 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28e851a392b94a6a8f9cb9df3437d7ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28e851a392b94a6a8f9cb9df3437d7ee, State: Initialized, Role: FOLLOWER
I20260812 11:09:23.499701 17736 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28e851a392b94a6a8f9cb9df3437d7ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28e851a392b94a6a8f9cb9df3437d7ee" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 40401 } }
I20260812 11:09:23.499897 17736 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28e851a392b94a6a8f9cb9df3437d7ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:09:23.500012 17736 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28e851a392b94a6a8f9cb9df3437d7ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:09:23.500154 17736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28e851a392b94a6a8f9cb9df3437d7ee [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:23.501632 17736 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28e851a392b94a6a8f9cb9df3437d7ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28e851a392b94a6a8f9cb9df3437d7ee" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 40401 } }
I20260812 11:09:23.502058 17736 leader_election.cc:302] T 00000000000000000000000000000000 P 28e851a392b94a6a8f9cb9df3437d7ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28e851a392b94a6a8f9cb9df3437d7ee; no voters:
I20260812 11:09:23.502444 17736 leader_election.cc:288] T 00000000000000000000000000000000 P 28e851a392b94a6a8f9cb9df3437d7ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:09:23.502965 17745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28e851a392b94a6a8f9cb9df3437d7ee [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:23.503316 17745 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28e851a392b94a6a8f9cb9df3437d7ee [term 1 LEADER]: Becoming Leader. State: Replica: 28e851a392b94a6a8f9cb9df3437d7ee, State: Running, Role: LEADER
Aug 12 11:09:23 dist-test-slave-drl0 krb5kdc[17596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:23.503830 17745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28e851a392b94a6a8f9cb9df3437d7ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28e851a392b94a6a8f9cb9df3437d7ee" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 40401 } }
I20260812 11:09:23.504256 17736 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28e851a392b94a6a8f9cb9df3437d7ee [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:09:23.507536 17746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28e851a392b94a6a8f9cb9df3437d7ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28e851a392b94a6a8f9cb9df3437d7ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28e851a392b94a6a8f9cb9df3437d7ee" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 40401 } } }
I20260812 11:09:23.507733 17746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28e851a392b94a6a8f9cb9df3437d7ee [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:23.508512 17748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28e851a392b94a6a8f9cb9df3437d7ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28e851a392b94a6a8f9cb9df3437d7ee. Latest consensus state: current_term: 1 leader_uuid: "28e851a392b94a6a8f9cb9df3437d7ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28e851a392b94a6a8f9cb9df3437d7ee" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 40401 } } }
I20260812 11:09:23.511313 17748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28e851a392b94a6a8f9cb9df3437d7ee [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:23.512411 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:23.481841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60022 (local address 127.17.44.254:40401)
0812 11:09:23.482768 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:23.482857 (+ 89us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:23.483864 (+ 1007us) server_negotiation.cc:407] Connection header received
0812 11:09:23.489322 (+ 5458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:23.489351 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:23.490350 (+ 999us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:23.491456 (+ 1106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:23.494210 (+ 2754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:23.495951 (+ 1741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:23.497440 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:23.498286 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:23.498489 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:23.504876 (+ 6387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:23.504888 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:23.504903 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:23.504939 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:23.507597 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:23.508335 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:23.508344 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:23.508351 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:23.508541 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:23.508985 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:23.508988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:23.508989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:23.509470 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:23.511006 (+ 1536us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:23.511181 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 11:09:23.511505 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":268,"threads_started":1}
I20260812 11:09:23.512732 17765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:09:23.516641 17765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:09:23.524529 17765 catalog_manager.cc:1347] Generated new cluster ID: 76217361dbe64a958b2d656912ec99b4
I20260812 11:09:23.524616 17765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:09:23.536829 17765 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:09:23.537904 17765 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:09:23.551538 17765 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28e851a392b94a6a8f9cb9df3437d7ee: Generated new TSK 0
I20260812 11:09:23.552865 17765 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
I20260812 11:09:23.590369 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:40401
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:46575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:23.760097 17780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:23.760346 17780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:23.760391 17780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:23.764452 17780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:23.764511 17780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:23.764569 17780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:09:23.769994 17780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:46575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:40401
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:23.771133 17780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:23.772186 17780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:23.783313 17788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:23.783424 17789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:23.783396 17792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:23.783802 17780 server_base.cc:1034] running on GCE node
I20260812 11:09:23.784181 17780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:23.784876 17780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:23.786073 17780 hybrid_clock.cc:648] HybridClock initialized: now 1786532963786059 us; error 26 us; skew 500 ppm
Aug 12 11:09:23 dist-test-slave-drl0 krb5kdc[17596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532963, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:23.789634 17780 init.cc:377] Logged in from keytab as kudu/127.17.44.193@KRBTEST.COM (short username kudu)
I20260812 11:09:23.790618 17780 webserver.cc:460] Webserver started at http://127.17.44.193:46079/ using document root <none> and password file <none>
I20260812 11:09:23.791289 17780 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:23.791366 17780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:23.791558 17780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:23.793715 17780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "c06d3d5618614593a33e684b57c67c8d"
format_stamp: "Formatted at 2026-08-12 11:09:23 on dist-test-slave-drl0"
server_key: "7191c928db973980ad9c7ea98369cc77"
server_key_iv: "082621e37d9f4a07c5e16e042ec4f95e"
server_key_version: "encryptionkey@0"
I20260812 11:09:23.794245 17780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "c06d3d5618614593a33e684b57c67c8d"
format_stamp: "Formatted at 2026-08-12 11:09:23 on dist-test-slave-drl0"
server_key: "7191c928db973980ad9c7ea98369cc77"
server_key_iv: "082621e37d9f4a07c5e16e042ec4f95e"
server_key_version: "encryptionkey@0"
I20260812 11:09:23.798538 17780 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260812 11:09:23.801743 17802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:23.803550 17780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:09:23.803710 17780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "c06d3d5618614593a33e684b57c67c8d"
format_stamp: "Formatted at 2026-08-12 11:09:23 on dist-test-slave-drl0"
server_key: "7191c928db973980ad9c7ea98369cc77"
server_key_iv: "082621e37d9f4a07c5e16e042ec4f95e"
server_key_version: "encryptionkey@0"
I20260812 11:09:23.803835 17780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:23.831033 17780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:23.834614 17780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:23.834872 17780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:23.835641 17780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:23.836582 17780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:23.836658 17780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:23.836733 17780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:23.836778 17780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:23.858276 17780 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:33287
I20260812 11:09:23.858299 17978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:33287 every 8 connection(s)
I20260812 11:09:23.859452 17780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:09:23.860112 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 17780
I20260812 11:09:23.860236 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:09:23.860512 17587 external_mini_cluster.cc:1428] Setting key 5bbbe302f1bd13aa87b65483a943e65d
WARNING: no policy specified for kudu/127.17.44.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.194@KRBTEST.COM" created.
Aug 12 11:09:23 dist-test-slave-drl0 krb5kdc[17596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532963, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:23.883881 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:23.863137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:47847 (local address 127.17.44.254:40401)
0812 11:09:23.863302 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:23.863306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:23.865341 (+ 2035us) server_negotiation.cc:407] Connection header received
0812 11:09:23.867607 (+ 2266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:23.867609 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:23.867666 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:23.867762 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:23.869889 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:23.870429 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:23.872646 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:23.872880 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:23.872967 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:23.878442 (+ 5475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:23.878448 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:23.878454 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:23.878492 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:23.880608 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:23.881530 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:23.881533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:23.881534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:23.881622 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:23.882205 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:23.882210 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:23.882211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:23.882578 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:23.882704 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:23.883612 (+ 908us) server_negotiation.cc:299] Negotiation successful
0812 11:09:23.883747 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 11:09:23.885931 17985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:23.863654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:40401 (local address 127.17.44.193:47847)
0812 11:09:23.864985 (+ 1331us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:23.865144 (+ 159us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:23.867318 (+ 2174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:23.867973 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:23.867985 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:23.868694 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:23.869664 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:23.869693 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:23.870606 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:23.870612 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:23.872492 (+ 1880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:23.872509 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:23.873055 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:23.873060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:23.873359 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:23.874102 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:23.874123 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:23.878242 (+ 4119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:23.880742 (+ 2500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:23.880749 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:23.880770 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:23.881239 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:23.881736 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:23.881742 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:23.881747 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:23.881951 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:23.882741 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:23.882751 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:23.883190 (+ 439us) client_negotiation.cc:769] Sending connection context
0812 11:09:23.883526 (+ 336us) client_negotiation.cc:240] Negotiation successful
0812 11:09:23.883984 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":945,"thread_start_us":268,"threads_started":1}
I20260812 11:09:23.887845 17980 heartbeater.cc:347] Connected to a master server at 127.17.44.254:40401
I20260812 11:09:23.888134 17980 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:23.888682 17980 heartbeater.cc:511] Master 127.17.44.254:40401 requested a full tablet report, sending...
I20260812 11:09:23.890856 17674 ts_manager.cc:194] Registered new tserver with Master: c06d3d5618614593a33e684b57c67c8d (127.17.44.193:33287)
I20260812 11:09:23.892565 17674 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.193@KRBTEST.COM'} at 127.17.44.193:47847
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
I20260812 11:09:23.900343 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:40401
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:46575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:24.070626 17988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:24.070976 17988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:24.071064 17988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:24.075285 17988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:24.075366 17988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:24.075457 17988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:09:24.082371 17988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:46575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:40401
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:24.083926 17988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:24.085260 17988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:24.097879 17998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:24.098042 17995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:24.097970 17994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:24.098416 17988 server_base.cc:1034] running on GCE node
I20260812 11:09:24.098876 17988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:24.099578 17988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:24.100804 17988 hybrid_clock.cc:648] HybridClock initialized: now 1786532964100758 us; error 69 us; skew 500 ppm
Aug 12 11:09:24 dist-test-slave-drl0 krb5kdc[17596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532964, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:24.104302 17988 init.cc:377] Logged in from keytab as kudu/127.17.44.194@KRBTEST.COM (short username kudu)
I20260812 11:09:24.105419 17988 webserver.cc:460] Webserver started at http://127.17.44.194:44227/ using document root <none> and password file <none>
I20260812 11:09:24.106323 17988 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:24.106436 17988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:24.106734 17988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:24.109295 17988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "379ff08071f54ee0a1cca23942564090"
format_stamp: "Formatted at 2026-08-12 11:09:24 on dist-test-slave-drl0"
server_key: "2bd888d608ebaffa6349318d63ac2376"
server_key_iv: "dcf58ab078d957b98e0c4241fc57ec76"
server_key_version: "encryptionkey@0"
I20260812 11:09:24.109925 17988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "379ff08071f54ee0a1cca23942564090"
format_stamp: "Formatted at 2026-08-12 11:09:24 on dist-test-slave-drl0"
server_key: "2bd888d608ebaffa6349318d63ac2376"
server_key_iv: "dcf58ab078d957b98e0c4241fc57ec76"
server_key_version: "encryptionkey@0"
I20260812 11:09:24.114082 17988 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 11:09:24.116801 18009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:24.118278 17988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 11:09:24.118468 17988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "379ff08071f54ee0a1cca23942564090"
format_stamp: "Formatted at 2026-08-12 11:09:24 on dist-test-slave-drl0"
server_key: "2bd888d608ebaffa6349318d63ac2376"
server_key_iv: "dcf58ab078d957b98e0c4241fc57ec76"
server_key_version: "encryptionkey@0"
I20260812 11:09:24.118587 17988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:24.152220 17988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:24.155802 17988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:24.156014 17988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:24.156635 17988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:24.157687 17988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:24.157771 17988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:24.157851 17988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:24.157896 17988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:24.182729 17988 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:41015
I20260812 11:09:24.182821 18183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:41015 every 8 connection(s)
I20260812 11:09:24.184110 17988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:09:24.193053 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 17988
I20260812 11:09:24.193229 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:09:24.193517 17587 external_mini_cluster.cc:1428] Setting key 01f2a2fc22c185d049631ba74986095c
Aug 12 11:09:24 dist-test-slave-drl0 krb5kdc[17596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532964, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:24.201694 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:24.186519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:55863 (local address 127.17.44.254:40401)
0812 11:09:24.186686 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:24.186690 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:24.188578 (+ 1888us) server_negotiation.cc:407] Connection header received
0812 11:09:24.189706 (+ 1128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:24.189709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:24.189763 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:24.189847 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:24.191661 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:24.192203 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:24.194235 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:24.194464 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:24.194578 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:24.197827 (+ 3249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:24.197831 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:24.197834 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:24.197868 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:24.199448 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:24.199964 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:24.199973 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:24.199974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:24.200046 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:24.200586 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:24.200589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:24.200591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:24.200789 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:24.200900 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:24.201399 (+ 499us) server_negotiation.cc:299] Negotiation successful
0812 11:09:24.201522 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 11:09:24.202733 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:24.187053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:40401 (local address 127.17.44.194:55863)
0812 11:09:24.188371 (+ 1318us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:24.188417 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:24.189450 (+ 1033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:24.190051 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:24.190058 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:24.190773 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:24.191479 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:24.191495 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:24.192351 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:24.192354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:24.194094 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:24.194104 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:24.194621 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:24.194625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:24.194853 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:24.195655 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:24.195682 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:24.197657 (+ 1975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:24.199572 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:24.199578 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:24.199592 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:24.199853 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:24.200139 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:24.200143 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:24.200145 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:24.200477 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:24.200897 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:24.200904 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:24.201150 (+ 246us) client_negotiation.cc:769] Sending connection context
0812 11:09:24.201340 (+ 190us) client_negotiation.cc:240] Negotiation successful
0812 11:09:24.201655 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1090,"thread_start_us":753,"threads_started":1}
I20260812 11:09:24.204236 18187 heartbeater.cc:347] Connected to a master server at 127.17.44.254:40401
I20260812 11:09:24.204499 18187 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:24.205041 18187 heartbeater.cc:511] Master 127.17.44.254:40401 requested a full tablet report, sending...
I20260812 11:09:24.206573 17674 ts_manager.cc:194] Registered new tserver with Master: 379ff08071f54ee0a1cca23942564090 (127.17.44.194:41015)
I20260812 11:09:24.207275 17674 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.194@KRBTEST.COM'} at 127.17.44.194:55863
WARNING: no policy specified for kudu/127.17.44.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
I20260812 11:09:24.231707 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:40401
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:46575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:24.398263 18200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:24.398599 18200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:24.398702 18200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:24.404234 18200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:24.404312 18200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:24.404418 18200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:09:24.410620 18200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:46575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:40401
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:24.411850 18200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:24.412945 18200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:24.425365 18211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:24.425624 18213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:24.425707 18210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:24.426000 18200 server_base.cc:1034] running on GCE node
I20260812 11:09:24.426494 18200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:24.427340 18200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:24.428577 18200 hybrid_clock.cc:648] HybridClock initialized: now 1786532964428520 us; error 65 us; skew 500 ppm
Aug 12 11:09:24 dist-test-slave-drl0 krb5kdc[17596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532964, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:24.431798 18200 init.cc:377] Logged in from keytab as kudu/127.17.44.195@KRBTEST.COM (short username kudu)
I20260812 11:09:24.432657 18200 webserver.cc:460] Webserver started at http://127.17.44.195:34765/ using document root <none> and password file <none>
I20260812 11:09:24.433238 18200 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:24.433329 18200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:24.433568 18200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:24.435995 18200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "a860498c8c434cca989c54723675b522"
format_stamp: "Formatted at 2026-08-12 11:09:24 on dist-test-slave-drl0"
server_key: "e855b05d512922868ccdfb56d9ff3e14"
server_key_iv: "3b4ab5c6ff9c0ee76637edf156d445fc"
server_key_version: "encryptionkey@0"
I20260812 11:09:24.436615 18200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "a860498c8c434cca989c54723675b522"
format_stamp: "Formatted at 2026-08-12 11:09:24 on dist-test-slave-drl0"
server_key: "e855b05d512922868ccdfb56d9ff3e14"
server_key_iv: "3b4ab5c6ff9c0ee76637edf156d445fc"
server_key_version: "encryptionkey@0"
I20260812 11:09:24.441665 18200 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 11:09:24.444964 18223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:24.446658 18200 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:09:24.446812 18200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "a860498c8c434cca989c54723675b522"
format_stamp: "Formatted at 2026-08-12 11:09:24 on dist-test-slave-drl0"
server_key: "e855b05d512922868ccdfb56d9ff3e14"
server_key_iv: "3b4ab5c6ff9c0ee76637edf156d445fc"
server_key_version: "encryptionkey@0"
I20260812 11:09:24.446935 18200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:24.488566 18200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:24.492936 18200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:24.493206 18200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:24.494050 18200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:24.495098 18200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:24.495169 18200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:24.495253 18200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:24.495297 18200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:24.517628 18200 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:42031
I20260812 11:09:24.518152 18422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:42031 every 8 connection(s)
I20260812 11:09:24.518702 18200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:09:24.521605 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 18200
I20260812 11:09:24.521785 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:09:24.522336 17587 external_mini_cluster.cc:1428] Setting key c27f9a777b0308aca6e7d17cf3d5143e
Aug 12 11:09:24 dist-test-slave-drl0 krb5kdc[17596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532964, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:24.536377 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:24.521133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:38399 (local address 127.17.44.254:40401)
0812 11:09:24.521451 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:24.521458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:24.522882 (+ 1424us) server_negotiation.cc:407] Connection header received
0812 11:09:24.524972 (+ 2090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:24.524976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:24.525049 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:24.525183 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:24.527153 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:24.527692 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:24.529023 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:24.529244 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:24.529338 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:24.532796 (+ 3458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:24.532799 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:24.532801 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:24.532825 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:24.534208 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:24.534779 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:24.534782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:24.534784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:24.534864 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:24.535318 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:24.535321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:24.535322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:24.535504 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:24.535585 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:24.536137 (+ 552us) server_negotiation.cc:299] Negotiation successful
0812 11:09:24.536241 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260812 11:09:24.537356 18427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:24.521529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:40401 (local address 127.17.44.195:38399)
0812 11:09:24.522668 (+ 1139us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:24.522731 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:24.524370 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:24.525441 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:24.525448 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:24.526206 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:24.526907 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:24.526984 (+ 77us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:24.527847 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:24.527850 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:24.528874 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:24.528885 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:24.529412 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:24.529415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:24.529617 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:24.530315 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:24.530332 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:24.532586 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:24.534339 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:24.534344 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:24.534356 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:24.534625 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:24.535027 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:24.535030 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:24.535032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:24.535180 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:24.535625 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:24.535630 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:24.535863 (+ 233us) client_negotiation.cc:769] Sending connection context
0812 11:09:24.536018 (+ 155us) client_negotiation.cc:240] Negotiation successful
0812 11:09:24.536278 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":237,"threads_started":1}
I20260812 11:09:24.539747 18423 heartbeater.cc:347] Connected to a master server at 127.17.44.254:40401
I20260812 11:09:24.540032 18423 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:24.540568 18423 heartbeater.cc:511] Master 127.17.44.254:40401 requested a full tablet report, sending...
I20260812 11:09:24.541918 17674 ts_manager.cc:194] Registered new tserver with Master: a860498c8c434cca989c54723675b522 (127.17.44.195:42031)
I20260812 11:09:24.542510 17674 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.195@KRBTEST.COM'} at 127.17.44.195:38399
I20260812 11:09:24.552033 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 11:09:24 dist-test-slave-drl0 krb5kdc[17596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532964, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 11:09:24 dist-test-slave-drl0 krb5kdc[17596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532964, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.44.193@KRBTEST.COM
I20260812 11:09:24.585901 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:24.574007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57880 (local address 127.17.44.193:33287)
0812 11:09:24.574347 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:24.574354 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:24.574381 (+ 27us) server_negotiation.cc:407] Connection header received
0812 11:09:24.574560 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:24.574567 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:24.574765 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:24.574892 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:24.575772 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:24.576967 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:24.577899 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:24.578345 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:24.578466 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:24.580985 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:24.580997 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:24.581015 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:24.581048 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:24.583588 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:24.584137 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:24.584143 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:24.584144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:24.584236 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:24.584674 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:24.584677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:24.584678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:24.584930 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:24.585105 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:24.585560 (+ 455us) server_negotiation.cc:299] Negotiation successful
0812 11:09:24.585697 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":117,"threads_started":1}
W20260812 11:09:24.587080 17844 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57880
Aug 12 11:09:24 dist-test-slave-drl0 krb5kdc[17596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 11:09:24 dist-test-slave-drl0 krb5kdc[17596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.193@KRBTEST.COM
I20260812 11:09:24.619655 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:24.610047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57888 (local address 127.17.44.193:33287)
0812 11:09:24.610247 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:24.610252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:24.610383 (+ 131us) server_negotiation.cc:407] Connection header received
0812 11:09:24.610566 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:24.610570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:24.610624 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:24.610713 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:24.611514 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:24.612055 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:24.612928 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:24.613170 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:24.613285 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:24.616069 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:24.616072 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:24.616074 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:24.616100 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:24.617544 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:24.617988 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:24.617991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:24.617992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:24.618071 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:24.618485 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:24.618491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:24.618492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:24.618806 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:24.619139 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:24.619315 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 11:09:24.619469 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 11:09:24.621080 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 17780
I20260812 11:09:24.633846 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 17988
I20260812 11:09:24.645717 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 18200
I20260812 11:09:24.653141 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 17616
2026-08-12T11:09:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2919 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:09:24 dist-test-slave-drl0 krb5kdc[18458](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:09:24 dist-test-slave-drl0 krb5kdc[18458](info): set up 2 sockets
Aug 12 11:09:24 dist-test-slave-drl0 krb5kdc[18458](info): commencing operation
krb5kdc: starting...
W20260812 11:09:25.951468 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:09:26 dist-test-slave-drl0 krb5kdc[18458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:09:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:09:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:09:26.091463 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:42651
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:33479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:42651
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:26.212069 18480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:26.212324 18480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:26.212363 18480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:26.216315 18480 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:09:26.216364 18480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:26.216384 18480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:26.216405 18480 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:09:26.216421 18480 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:09:26.221596 18480 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:33479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:42651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:42651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:26.222544 18480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:26.223893 18480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:26.234524 18491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:26.234534 18499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:26.234572 18493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:26.234908 18480 server_base.cc:1034] running on GCE node
I20260812 11:09:26.235707 18480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:26.236933 18480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:26.238166 18480 hybrid_clock.cc:648] HybridClock initialized: now 1786532966238132 us; error 48 us; skew 500 ppm
Aug 12 11:09:26 dist-test-slave-drl0 krb5kdc[18458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532966, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:26.241302 18480 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:09:26.242225 18480 webserver.cc:460] Webserver started at http://127.17.44.254:44611/ using document root <none> and password file <none>
I20260812 11:09:26.242771 18480 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:26.242830 18480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:26.243064 18480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:26.245091 18480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "4a539a04266a46b88dfda233b8a0d42d"
format_stamp: "Formatted at 2026-08-12 11:09:26 on dist-test-slave-drl0"
server_key: "55c97d8822484275d66ccd9cf8385480"
server_key_iv: "4cd8b7959f14a69dd24a6538d47da84d"
server_key_version: "encryptionkey@0"
I20260812 11:09:26.245591 18480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "4a539a04266a46b88dfda233b8a0d42d"
format_stamp: "Formatted at 2026-08-12 11:09:26 on dist-test-slave-drl0"
server_key: "55c97d8822484275d66ccd9cf8385480"
server_key_iv: "4cd8b7959f14a69dd24a6538d47da84d"
server_key_version: "encryptionkey@0"
I20260812 11:09:26.250345 18480 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260812 11:09:26.253501 18509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:26.255297 18480 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:09:26.255456 18480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "4a539a04266a46b88dfda233b8a0d42d"
format_stamp: "Formatted at 2026-08-12 11:09:26 on dist-test-slave-drl0"
server_key: "55c97d8822484275d66ccd9cf8385480"
server_key_iv: "4cd8b7959f14a69dd24a6538d47da84d"
server_key_version: "encryptionkey@0"
I20260812 11:09:26.255574 18480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:26.265769 18480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:26.269151 18480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:26.269317 18480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:26.277403 18480 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:42651
I20260812 11:09:26.277431 18586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:42651 every 8 connection(s)
I20260812 11:09:26.278785 18480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:09:26.279208 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 18480
I20260812 11:09:26.279340 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:09:26.279883 17587 external_mini_cluster.cc:1428] Setting key 7fe357a20862685ffc46e7b6d2127eaa
I20260812 11:09:26.282469 18587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:26.290524 18587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a539a04266a46b88dfda233b8a0d42d: Bootstrap starting.
Aug 12 11:09:26 dist-test-slave-drl0 krb5kdc[18458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:26.292799 18587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a539a04266a46b88dfda233b8a0d42d: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:26.293565 18587 log.cc:824] T 00000000000000000000000000000000 P 4a539a04266a46b88dfda233b8a0d42d: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:26.296001 18587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a539a04266a46b88dfda233b8a0d42d: No bootstrap required, opened a new log
I20260812 11:09:26.297580 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:26.281547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60834 (local address 127.17.44.254:42651)
0812 11:09:26.281972 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:26.281998 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:26.282033 (+ 35us) server_negotiation.cc:407] Connection header received
0812 11:09:26.283233 (+ 1200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:26.283261 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:26.283737 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:26.284138 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:26.285070 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:26.286103 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:26.287582 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:26.288047 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:26.288179 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:26.291542 (+ 3363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:26.291547 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:26.291561 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:26.291589 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:26.293411 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:26.293916 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:26.293925 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:26.293934 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:26.294080 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:26.294797 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:26.294802 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:26.294803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:26.295628 (+ 825us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:26.295792 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:26.296285 (+ 493us) server_negotiation.cc:299] Negotiation successful
0812 11:09:26.296578 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":150,"threads_started":1}
I20260812 11:09:26.299436 18587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a539a04266a46b88dfda233b8a0d42d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a539a04266a46b88dfda233b8a0d42d" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42651 } }
I20260812 11:09:26.299707 18587 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a539a04266a46b88dfda233b8a0d42d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:26.299758 18587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a539a04266a46b88dfda233b8a0d42d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a539a04266a46b88dfda233b8a0d42d, State: Initialized, Role: FOLLOWER
I20260812 11:09:26.300252 18587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a539a04266a46b88dfda233b8a0d42d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a539a04266a46b88dfda233b8a0d42d" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42651 } }
I20260812 11:09:26.300439 18587 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a539a04266a46b88dfda233b8a0d42d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:09:26.300493 18587 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a539a04266a46b88dfda233b8a0d42d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:09:26.300567 18587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a539a04266a46b88dfda233b8a0d42d [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:26.301571 18587 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a539a04266a46b88dfda233b8a0d42d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a539a04266a46b88dfda233b8a0d42d" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42651 } }
I20260812 11:09:26.301869 18587 leader_election.cc:302] T 00000000000000000000000000000000 P 4a539a04266a46b88dfda233b8a0d42d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a539a04266a46b88dfda233b8a0d42d; no voters:
I20260812 11:09:26.302163 18587 leader_election.cc:288] T 00000000000000000000000000000000 P 4a539a04266a46b88dfda233b8a0d42d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:09:26.302349 18594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a539a04266a46b88dfda233b8a0d42d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:26.302697 18594 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a539a04266a46b88dfda233b8a0d42d [term 1 LEADER]: Becoming Leader. State: Replica: 4a539a04266a46b88dfda233b8a0d42d, State: Running, Role: LEADER
I20260812 11:09:26.303200 18594 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a539a04266a46b88dfda233b8a0d42d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a539a04266a46b88dfda233b8a0d42d" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42651 } }
I20260812 11:09:26.304198 18587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a539a04266a46b88dfda233b8a0d42d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:09:26.305122 18596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a539a04266a46b88dfda233b8a0d42d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a539a04266a46b88dfda233b8a0d42d. Latest consensus state: current_term: 1 leader_uuid: "4a539a04266a46b88dfda233b8a0d42d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a539a04266a46b88dfda233b8a0d42d" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42651 } } }
I20260812 11:09:26.305127 18595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a539a04266a46b88dfda233b8a0d42d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a539a04266a46b88dfda233b8a0d42d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a539a04266a46b88dfda233b8a0d42d" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42651 } } }
I20260812 11:09:26.305291 18596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a539a04266a46b88dfda233b8a0d42d [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:26.305308 18595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a539a04266a46b88dfda233b8a0d42d [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:26.305796 18602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:09:26.309005 18602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:09:26.315407 18602 catalog_manager.cc:1347] Generated new cluster ID: bede10fb257941b19b1765c90dd18010
I20260812 11:09:26.315481 18602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:09:26.335430 18602 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:09:26.336450 18602 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:09:26.345461 18602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a539a04266a46b88dfda233b8a0d42d: Generated new TSK 0
I20260812 11:09:26.346765 18602 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
I20260812 11:09:26.382584 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:42651
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:33479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:26.538048 18627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:26.538417 18627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:26.538470 18627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:26.544062 18627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:26.544150 18627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:26.544265 18627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:09:26.549708 18627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:33479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:42651
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:26.550812 18627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:26.551914 18627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:26.563098 18633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:26.563124 18634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:26.563098 18637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:26.563295 18627 server_base.cc:1034] running on GCE node
I20260812 11:09:26.563755 18627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:26.564415 18627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:26.565625 18627 hybrid_clock.cc:648] HybridClock initialized: now 1786532966565594 us; error 42 us; skew 500 ppm
Aug 12 11:09:26 dist-test-slave-drl0 krb5kdc[18458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532966, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:26.568747 18627 init.cc:377] Logged in from keytab as kudu/127.17.44.193@KRBTEST.COM (short username kudu)
I20260812 11:09:26.569578 18627 webserver.cc:460] Webserver started at http://127.17.44.193:37271/ using document root <none> and password file <none>
I20260812 11:09:26.570154 18627 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:26.570248 18627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:26.570478 18627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:26.572573 18627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "0f6c507c4ea4437889dff506e520f7fd"
format_stamp: "Formatted at 2026-08-12 11:09:26 on dist-test-slave-drl0"
server_key: "bc35ba6814124e463278b56bbf50cf39"
server_key_iv: "35c0bb26acb779c28b0e015b9d246e2f"
server_key_version: "encryptionkey@0"
I20260812 11:09:26.573132 18627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "0f6c507c4ea4437889dff506e520f7fd"
format_stamp: "Formatted at 2026-08-12 11:09:26 on dist-test-slave-drl0"
server_key: "bc35ba6814124e463278b56bbf50cf39"
server_key_iv: "35c0bb26acb779c28b0e015b9d246e2f"
server_key_version: "encryptionkey@0"
I20260812 11:09:26.577086 18627 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260812 11:09:26.580499 18648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:26.582414 18627 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 11:09:26.582648 18627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "0f6c507c4ea4437889dff506e520f7fd"
format_stamp: "Formatted at 2026-08-12 11:09:26 on dist-test-slave-drl0"
server_key: "bc35ba6814124e463278b56bbf50cf39"
server_key_iv: "35c0bb26acb779c28b0e015b9d246e2f"
server_key_version: "encryptionkey@0"
I20260812 11:09:26.582828 18627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:26.597496 18627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:26.600934 18627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:26.601198 18627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:26.601819 18627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:26.602748 18627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:26.602839 18627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:26.602916 18627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:26.603024 18627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260812 11:09:26.613941 18627 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:39643
I20260812 11:09:26.613996 18820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:39643 every 8 connection(s)
I20260812 11:09:26.614854 18627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:09:26.621646 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 18627
I20260812 11:09:26.621744 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:09:26.622009 17587 external_mini_cluster.cc:1428] Setting key 961f90423e38646c18529f41957ae513
Aug 12 11:09:26 dist-test-slave-drl0 krb5kdc[18458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532966, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:26.630330 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:26.616986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:56033 (local address 127.17.44.254:42651)
0812 11:09:26.617177 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:26.617181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:26.617885 (+ 704us) server_negotiation.cc:407] Connection header received
0812 11:09:26.618704 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:26.618707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:26.618757 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:26.618840 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:26.620682 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:26.621249 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:26.622850 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:26.623224 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:26.623354 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:26.626251 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:26.626255 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:26.626257 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:26.626281 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:26.627840 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:26.628396 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:26.628400 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:26.628402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:26.628495 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:26.628843 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:26.628848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:26.628850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:26.629045 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:26.629170 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:26.629656 (+ 486us) server_negotiation.cc:299] Negotiation successful
0812 11:09:26.630032 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 11:09:26.631577 18826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:26.617253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42651 (local address 127.17.44.193:56033)
0812 11:09:26.617720 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:26.617761 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:26.618499 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:26.619106 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:26.619113 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:26.619468 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:26.620427 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:26.620450 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:26.621445 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:26.621449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:26.622683 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:26.622696 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:26.623425 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:26.623429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:26.623619 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:26.624306 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:26.624323 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:26.626048 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:26.627991 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:26.627997 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:26.628012 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:26.628284 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:26.628593 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:26.628597 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:26.628599 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:26.628741 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:26.629157 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:26.629163 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:26.629423 (+ 260us) client_negotiation.cc:769] Sending connection context
0812 11:09:26.629612 (+ 189us) client_negotiation.cc:240] Negotiation successful
0812 11:09:26.630250 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":119,"threads_started":1}
WARNING: no policy specified for kudu/127.17.44.194@KRBTEST.COM; defaulting to no policy
I20260812 11:09:26.633077 18821 heartbeater.cc:347] Connected to a master server at 127.17.44.254:42651
I20260812 11:09:26.633343 18821 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:26.634047 18821 heartbeater.cc:511] Master 127.17.44.254:42651 requested a full tablet report, sending...
I20260812 11:09:26.635713 18534 ts_manager.cc:194] Registered new tserver with Master: 0f6c507c4ea4437889dff506e520f7fd (127.17.44.193:39643)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.194@KRBTEST.COM" created.
I20260812 11:09:26.637650 18534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.193@KRBTEST.COM'} at 127.17.44.193:56033
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
I20260812 11:09:26.655508 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:42651
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:33479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:26.775887 18832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:26.776144 18832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:26.776188 18832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:26.780396 18832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:26.780462 18832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:26.780524 18832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:09:26.786116 18832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:33479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:42651
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:26.787389 18832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:26.788471 18832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:26.800400 18846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:26.800387 18843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:26.800379 18832 server_base.cc:1034] running on GCE node
W20260812 11:09:26.800357 18840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:26.800978 18832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:26.801597 18832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:26.802835 18832 hybrid_clock.cc:648] HybridClock initialized: now 1786532966802773 us; error 73 us; skew 500 ppm
Aug 12 11:09:26 dist-test-slave-drl0 krb5kdc[18458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532966, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:26.806324 18832 init.cc:377] Logged in from keytab as kudu/127.17.44.194@KRBTEST.COM (short username kudu)
I20260812 11:09:26.807682 18832 webserver.cc:460] Webserver started at http://127.17.44.194:36871/ using document root <none> and password file <none>
I20260812 11:09:26.808292 18832 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:26.808385 18832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:26.808612 18832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:26.810632 18832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "26f75e5f673641c98de4d403c5f81f0d"
format_stamp: "Formatted at 2026-08-12 11:09:26 on dist-test-slave-drl0"
server_key: "08f5d2c0dbc25a02c0ce944281286227"
server_key_iv: "3551091a4bcce8814757397bc898e36e"
server_key_version: "encryptionkey@0"
I20260812 11:09:26.811218 18832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "26f75e5f673641c98de4d403c5f81f0d"
format_stamp: "Formatted at 2026-08-12 11:09:26 on dist-test-slave-drl0"
server_key: "08f5d2c0dbc25a02c0ce944281286227"
server_key_iv: "3551091a4bcce8814757397bc898e36e"
server_key_version: "encryptionkey@0"
I20260812 11:09:26.815276 18832 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 11:09:26.817715 18855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:26.819154 18832 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:09:26.819293 18832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "26f75e5f673641c98de4d403c5f81f0d"
format_stamp: "Formatted at 2026-08-12 11:09:26 on dist-test-slave-drl0"
server_key: "08f5d2c0dbc25a02c0ce944281286227"
server_key_iv: "3551091a4bcce8814757397bc898e36e"
server_key_version: "encryptionkey@0"
I20260812 11:09:26.819411 18832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:26.857232 18832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:26.861443 18832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:26.861672 18832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:26.862546 18832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:26.863659 18832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:26.863739 18832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:26.863814 18832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:26.863852 18832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:26.874132 18832 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:46653
I20260812 11:09:26.874174 19043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:46653 every 8 connection(s)
I20260812 11:09:26.875097 18832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:09:26.884047 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 18832
I20260812 11:09:26.884182 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:09:26.884483 17587 external_mini_cluster.cc:1428] Setting key 22dff8eaf1e87028eae4be68ab02480d
Aug 12 11:09:26 dist-test-slave-drl0 krb5kdc[18458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532966, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:26.890534 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:26.877065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:44757 (local address 127.17.44.254:42651)
0812 11:09:26.877231 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:26.877235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:26.878281 (+ 1046us) server_negotiation.cc:407] Connection header received
0812 11:09:26.879236 (+ 955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:26.879239 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:26.879300 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:26.879385 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:26.880612 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:26.881157 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:26.882473 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:26.882717 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:26.882810 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:26.885629 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:26.885633 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:26.885636 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:26.885666 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:26.887597 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:26.888294 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:26.888300 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:26.888302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:26.888409 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:26.888839 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:26.888845 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:26.888848 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:26.889106 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:26.889631 (+ 525us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:26.890139 (+ 508us) server_negotiation.cc:299] Negotiation successful
0812 11:09:26.890283 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 11:09:26.891278 19048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:26.877407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42651 (local address 127.17.44.194:44757)
0812 11:09:26.878110 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:26.878149 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:26.878915 (+ 766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:26.879574 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:26.879581 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:26.879938 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:26.880446 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:26.880459 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:26.881302 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:26.881305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:26.882324 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:26.882335 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:26.883029 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:26.883033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:26.883174 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:26.883726 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:26.883742 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:26.885461 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:26.887710 (+ 2249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:26.887720 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:26.887738 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:26.888138 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:26.888513 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:26.888516 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:26.888517 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:26.888649 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:26.889635 (+ 986us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:26.889641 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:26.889879 (+ 238us) client_negotiation.cc:769] Sending connection context
0812 11:09:26.890039 (+ 160us) client_negotiation.cc:240] Negotiation successful
0812 11:09:26.890293 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":113,"threads_started":1}
I20260812 11:09:26.892557 19045 heartbeater.cc:347] Connected to a master server at 127.17.44.254:42651
I20260812 11:09:26.892827 19045 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:26.893450 19045 heartbeater.cc:511] Master 127.17.44.254:42651 requested a full tablet report, sending...
I20260812 11:09:26.894761 18534 ts_manager.cc:194] Registered new tserver with Master: 26f75e5f673641c98de4d403c5f81f0d (127.17.44.194:46653)
I20260812 11:09:26.895426 18534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.194@KRBTEST.COM'} at 127.17.44.194:44757
WARNING: no policy specified for kudu/127.17.44.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
I20260812 11:09:26.919032 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:42651
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:33479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:27.051013 19059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:27.051278 19059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:27.051329 19059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:27.055543 19059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:27.055940 19059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:27.056047 19059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:09:27.061664 19059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:33479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:42651
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:27.062711 19059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:27.063781 19059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:27.074186 19076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:27.074199 19071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:27.074186 19072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:27.074604 19059 server_base.cc:1034] running on GCE node
I20260812 11:09:27.075218 19059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:27.075862 19059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:27.077046 19059 hybrid_clock.cc:648] HybridClock initialized: now 1786532967077026 us; error 32 us; skew 500 ppm
Aug 12 11:09:27 dist-test-slave-drl0 krb5kdc[18458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532967, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:27.080157 19059 init.cc:377] Logged in from keytab as kudu/127.17.44.195@KRBTEST.COM (short username kudu)
I20260812 11:09:27.081038 19059 webserver.cc:460] Webserver started at http://127.17.44.195:43617/ using document root <none> and password file <none>
I20260812 11:09:27.081609 19059 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:27.081676 19059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:27.081899 19059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:27.084213 19059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "c4f14a29429c424983a63a24c90ce430"
format_stamp: "Formatted at 2026-08-12 11:09:27 on dist-test-slave-drl0"
server_key: "93e40774a3ebb9da044617e5cd84c071"
server_key_iv: "00d56107e9f3f34e8a0bb480c0087db9"
server_key_version: "encryptionkey@0"
I20260812 11:09:27.084774 19059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "c4f14a29429c424983a63a24c90ce430"
format_stamp: "Formatted at 2026-08-12 11:09:27 on dist-test-slave-drl0"
server_key: "93e40774a3ebb9da044617e5cd84c071"
server_key_iv: "00d56107e9f3f34e8a0bb480c0087db9"
server_key_version: "encryptionkey@0"
I20260812 11:09:27.090864 19059 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260812 11:09:27.094410 19087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:27.095827 19059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 11:09:27.095932 19059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "c4f14a29429c424983a63a24c90ce430"
format_stamp: "Formatted at 2026-08-12 11:09:27 on dist-test-slave-drl0"
server_key: "93e40774a3ebb9da044617e5cd84c071"
server_key_iv: "00d56107e9f3f34e8a0bb480c0087db9"
server_key_version: "encryptionkey@0"
I20260812 11:09:27.096077 19059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:27.110796 19059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:27.114888 19059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:27.115298 19059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:27.116271 19059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:27.117172 19059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:27.117221 19059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:27.117287 19059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:27.117321 19059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:27.127539 19059 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:46839
I20260812 11:09:27.127563 19247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:46839 every 8 connection(s)
I20260812 11:09:27.128412 19059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:09:27.136560 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 19059
I20260812 11:09:27.136703 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:09:27.136983 17587 external_mini_cluster.cc:1428] Setting key b9ce2d5e89c193f02e6c3dcfe7aeea5b
Aug 12 11:09:27 dist-test-slave-drl0 krb5kdc[18458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532967, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:27.143755 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:27.130830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:47151 (local address 127.17.44.254:42651)
0812 11:09:27.131038 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:27.131043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:27.132216 (+ 1173us) server_negotiation.cc:407] Connection header received
0812 11:09:27.133247 (+ 1031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:27.133250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:27.133308 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:27.133418 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:27.134544 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.135132 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.136255 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.136598 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.136725 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:27.139804 (+ 3079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:27.139807 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:27.139812 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:27.139840 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:27.141318 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:27.141952 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:27.141958 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:27.141962 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:27.142047 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:27.142407 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:27.142410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:27.142412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:27.142649 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:27.142769 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:27.143460 (+ 691us) server_negotiation.cc:299] Negotiation successful
0812 11:09:27.143583 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 11:09:27.144582 19251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:27.131367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42651 (local address 127.17.44.195:47151)
0812 11:09:27.132052 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:27.132091 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:27.133026 (+ 935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:27.133584 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:27.133591 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:27.133931 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:27.134392 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:27.134405 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.135256 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.135259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:27.136124 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:27.136133 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.136794 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.136801 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:27.136968 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:27.137598 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:27.137614 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:27.139621 (+ 2007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:27.141540 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:27.141545 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:27.141567 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:27.141840 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:27.142147 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:27.142151 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:27.142153 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:27.142293 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:27.142769 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:27.142777 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:27.143083 (+ 306us) client_negotiation.cc:769] Sending connection context
0812 11:09:27.143413 (+ 330us) client_negotiation.cc:240] Negotiation successful
0812 11:09:27.143726 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":127,"threads_started":1}
I20260812 11:09:27.145787 19249 heartbeater.cc:347] Connected to a master server at 127.17.44.254:42651
I20260812 11:09:27.146005 19249 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:27.146438 19249 heartbeater.cc:511] Master 127.17.44.254:42651 requested a full tablet report, sending...
I20260812 11:09:27.147652 18534 ts_manager.cc:194] Registered new tserver with Master: c4f14a29429c424983a63a24c90ce430 (127.17.44.195:46839)
I20260812 11:09:27.148624 18534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.195@KRBTEST.COM'} at 127.17.44.195:47151
I20260812 11:09:27.151957 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:09:27.165433 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:27.157064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60850 (local address 127.17.44.254:42651)
0812 11:09:27.157207 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:27.157210 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:27.157478 (+ 268us) server_negotiation.cc:407] Connection header received
0812 11:09:27.157562 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:27.157564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:27.157605 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:27.157688 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:27.158632 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.159257 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.160120 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.160378 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.160504 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:27.161879 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:27.161884 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:27.161892 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:27.161915 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:27.163540 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:27.164019 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:27.164024 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:27.164026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:27.164117 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:27.164446 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:27.164450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:27.164452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:27.164685 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:27.164847 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:27.165121 (+ 274us) server_negotiation.cc:299] Negotiation successful
0812 11:09:27.165255 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 11:09:27.171829 18534 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:09:27.175532 18534 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:09:27.193768 19271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:27.187028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:46839 (local address 127.0.0.1:56398)
0812 11:09:27.188192 (+ 1164us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:27.188246 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:27.188410 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:27.189062 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:27.189069 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:27.189097 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:09:27.189242 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:27.189253 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.190473 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.190477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:27.192025 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:27.192037 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.193151 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.193156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:27.193287 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:27.193345 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 11:09:27.193462 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 11:09:27.193584 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":57,"thread_start_us":75,"threads_started":1}
I20260812 11:09:27.193775 19274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:27.187206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56398 (local address 127.17.44.195:46839)
0812 11:09:27.187801 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:27.187810 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:27.188564 (+ 754us) server_negotiation.cc:407] Connection header received
0812 11:09:27.188653 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:27.188660 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:27.188887 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:27.189047 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:09:27.189388 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.190337 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.192158 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.192999 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.193111 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:27.193327 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:27.193511 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 11:09:27.193615 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":111,"threads_started":1}
I20260812 11:09:27.194020 19278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:27.188294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:39643 (local address 127.0.0.1:55508)
0812 11:09:27.189333 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:27.189348 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:27.189432 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:27.189907 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:27.189910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:27.189930 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:09:27.190009 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:27.190015 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.191270 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.191275 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:27.192452 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:27.192463 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.193495 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.193498 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:27.193585 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:27.193606 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 11:09:27.193677 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 11:09:27.193729 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":956,"mutex_wait_us":1,"thread_start_us":98,"threads_started":1}
I20260812 11:09:27.194213 19275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:27.187923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:46653 (local address 127.0.0.1:34618)
0812 11:09:27.188440 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:27.188454 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:27.188530 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:27.189310 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:27.189316 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:27.189337 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:09:27.189441 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:27.189447 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.190861 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.190864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:27.192030 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:27.192037 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.193746 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.193750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:27.193854 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:27.193887 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 11:09:27.193964 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 11:09:27.194036 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"mutex_wait_us":23,"thread_start_us":78,"threads_started":1}
I20260812 11:09:27.194675 19276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:27.187246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34618 (local address 127.17.44.194:46653)
0812 11:09:27.188757 (+ 1511us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:27.188769 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:27.188789 (+ 20us) server_negotiation.cc:407] Connection header received
0812 11:09:27.188870 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:27.188875 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:27.189035 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:27.189163 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:09:27.189555 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.190712 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.192617 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.193455 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.193560 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:27.193680 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:27.194442 (+ 762us) server_negotiation.cc:299] Negotiation successful
0812 11:09:27.194523 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1417,"thread_start_us":923,"threads_started":1}
I20260812 11:09:27.195487 19280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:27.188445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55508 (local address 127.17.44.193:39643)
0812 11:09:27.189461 (+ 1016us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:27.189469 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:27.189493 (+ 24us) server_negotiation.cc:407] Connection header received
0812 11:09:27.189566 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:27.189571 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:27.189746 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:27.189903 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:09:27.190129 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.191158 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.192601 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.193185 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.193293 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:27.193431 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:27.195211 (+ 1780us) server_negotiation.cc:299] Negotiation successful
0812 11:09:27.195335 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":919,"thread_start_us":107,"threads_started":1}
I20260812 11:09:27.197330 19155 tablet_service.cc:1467] Processing CreateTablet for tablet 5d9378f261e245e0a129ece0c12f32d0 (DEFAULT_TABLE table=test-table [id=cdd9475bc2514671bfed292ca3392515]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:27.197510 18932 tablet_service.cc:1467] Processing CreateTablet for tablet 5d9378f261e245e0a129ece0c12f32d0 (DEFAULT_TABLE table=test-table [id=cdd9475bc2514671bfed292ca3392515]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:27.198415 19155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d9378f261e245e0a129ece0c12f32d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:27.198058 18720 tablet_service.cc:1467] Processing CreateTablet for tablet 5d9378f261e245e0a129ece0c12f32d0 (DEFAULT_TABLE table=test-table [id=cdd9475bc2514671bfed292ca3392515]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:27.198583 18932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d9378f261e245e0a129ece0c12f32d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:27.199013 18720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d9378f261e245e0a129ece0c12f32d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:27.205454 19281 tablet_bootstrap.cc:492] T 5d9378f261e245e0a129ece0c12f32d0 P c4f14a29429c424983a63a24c90ce430: Bootstrap starting.
I20260812 11:09:27.207278 19282 tablet_bootstrap.cc:492] T 5d9378f261e245e0a129ece0c12f32d0 P 0f6c507c4ea4437889dff506e520f7fd: Bootstrap starting.
I20260812 11:09:27.207296 19284 tablet_bootstrap.cc:492] T 5d9378f261e245e0a129ece0c12f32d0 P 26f75e5f673641c98de4d403c5f81f0d: Bootstrap starting.
I20260812 11:09:27.208360 19281 tablet_bootstrap.cc:654] T 5d9378f261e245e0a129ece0c12f32d0 P c4f14a29429c424983a63a24c90ce430: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:27.210348 19281 log.cc:824] T 5d9378f261e245e0a129ece0c12f32d0 P c4f14a29429c424983a63a24c90ce430: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:27.210664 19282 tablet_bootstrap.cc:654] T 5d9378f261e245e0a129ece0c12f32d0 P 0f6c507c4ea4437889dff506e520f7fd: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:27.211627 19284 tablet_bootstrap.cc:654] T 5d9378f261e245e0a129ece0c12f32d0 P 26f75e5f673641c98de4d403c5f81f0d: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:27.212401 19281 tablet_bootstrap.cc:492] T 5d9378f261e245e0a129ece0c12f32d0 P c4f14a29429c424983a63a24c90ce430: No bootstrap required, opened a new log
I20260812 11:09:27.212615 19281 ts_tablet_manager.cc:1397] T 5d9378f261e245e0a129ece0c12f32d0 P c4f14a29429c424983a63a24c90ce430: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260812 11:09:27.212810 19282 log.cc:824] T 5d9378f261e245e0a129ece0c12f32d0 P 0f6c507c4ea4437889dff506e520f7fd: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:27.213079 19284 log.cc:824] T 5d9378f261e245e0a129ece0c12f32d0 P 26f75e5f673641c98de4d403c5f81f0d: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:27.216274 19284 tablet_bootstrap.cc:492] T 5d9378f261e245e0a129ece0c12f32d0 P 26f75e5f673641c98de4d403c5f81f0d: No bootstrap required, opened a new log
I20260812 11:09:27.216559 19284 ts_tablet_manager.cc:1397] T 5d9378f261e245e0a129ece0c12f32d0 P 26f75e5f673641c98de4d403c5f81f0d: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260812 11:09:27.216859 19282 tablet_bootstrap.cc:492] T 5d9378f261e245e0a129ece0c12f32d0 P 0f6c507c4ea4437889dff506e520f7fd: No bootstrap required, opened a new log
I20260812 11:09:27.217291 19282 ts_tablet_manager.cc:1397] T 5d9378f261e245e0a129ece0c12f32d0 P 0f6c507c4ea4437889dff506e520f7fd: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260812 11:09:27.217760 19281 raft_consensus.cc:348] T 5d9378f261e245e0a129ece0c12f32d0 P c4f14a29429c424983a63a24c90ce430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f14a29429c424983a63a24c90ce430" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46839 } } peers { permanent_uuid: "26f75e5f673641c98de4d403c5f81f0d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46653 } } peers { permanent_uuid: "0f6c507c4ea4437889dff506e520f7fd" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39643 } }
I20260812 11:09:27.218070 19281 raft_consensus.cc:374] T 5d9378f261e245e0a129ece0c12f32d0 P c4f14a29429c424983a63a24c90ce430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:27.218128 19281 raft_consensus.cc:729] T 5d9378f261e245e0a129ece0c12f32d0 P c4f14a29429c424983a63a24c90ce430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4f14a29429c424983a63a24c90ce430, State: Initialized, Role: FOLLOWER
I20260812 11:09:27.218587 19281 consensus_queue.cc:260] T 5d9378f261e245e0a129ece0c12f32d0 P c4f14a29429c424983a63a24c90ce430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f14a29429c424983a63a24c90ce430" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46839 } } peers { permanent_uuid: "26f75e5f673641c98de4d403c5f81f0d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46653 } } peers { permanent_uuid: "0f6c507c4ea4437889dff506e520f7fd" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39643 } }
I20260812 11:09:27.219492 19249 heartbeater.cc:503] Master 127.17.44.254:42651 was elected leader, sending a full tablet report...
I20260812 11:09:27.220880 19289 raft_consensus.cc:482] T 5d9378f261e245e0a129ece0c12f32d0 P c4f14a29429c424983a63a24c90ce430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:09:27.221082 19289 raft_consensus.cc:504] T 5d9378f261e245e0a129ece0c12f32d0 P c4f14a29429c424983a63a24c90ce430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f14a29429c424983a63a24c90ce430" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46839 } } peers { permanent_uuid: "26f75e5f673641c98de4d403c5f81f0d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46653 } } peers { permanent_uuid: "0f6c507c4ea4437889dff506e520f7fd" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39643 } }
I20260812 11:09:27.222460 19289 leader_election.cc:288] T 5d9378f261e245e0a129ece0c12f32d0 P c4f14a29429c424983a63a24c90ce430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 26f75e5f673641c98de4d403c5f81f0d (127.17.44.194:46653), 0f6c507c4ea4437889dff506e520f7fd (127.17.44.193:39643)
I20260812 11:09:27.223412 19281 ts_tablet_manager.cc:1428] T 5d9378f261e245e0a129ece0c12f32d0 P c4f14a29429c424983a63a24c90ce430: Time spent starting tablet: real 0.011s user 0.006s sys 0.003s
I20260812 11:09:27.225733 19284 raft_consensus.cc:348] T 5d9378f261e245e0a129ece0c12f32d0 P 26f75e5f673641c98de4d403c5f81f0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f14a29429c424983a63a24c90ce430" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46839 } } peers { permanent_uuid: "26f75e5f673641c98de4d403c5f81f0d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46653 } } peers { permanent_uuid: "0f6c507c4ea4437889dff506e520f7fd" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39643 } }
I20260812 11:09:27.226083 19284 raft_consensus.cc:374] T 5d9378f261e245e0a129ece0c12f32d0 P 26f75e5f673641c98de4d403c5f81f0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:27.226138 19284 raft_consensus.cc:729] T 5d9378f261e245e0a129ece0c12f32d0 P 26f75e5f673641c98de4d403c5f81f0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26f75e5f673641c98de4d403c5f81f0d, State: Initialized, Role: FOLLOWER
I20260812 11:09:27.226183 19282 raft_consensus.cc:348] T 5d9378f261e245e0a129ece0c12f32d0 P 0f6c507c4ea4437889dff506e520f7fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f14a29429c424983a63a24c90ce430" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46839 } } peers { permanent_uuid: "26f75e5f673641c98de4d403c5f81f0d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46653 } } peers { permanent_uuid: "0f6c507c4ea4437889dff506e520f7fd" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39643 } }
I20260812 11:09:27.226419 19282 raft_consensus.cc:374] T 5d9378f261e245e0a129ece0c12f32d0 P 0f6c507c4ea4437889dff506e520f7fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:27.226469 19282 raft_consensus.cc:729] T 5d9378f261e245e0a129ece0c12f32d0 P 0f6c507c4ea4437889dff506e520f7fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f6c507c4ea4437889dff506e520f7fd, State: Initialized, Role: FOLLOWER
I20260812 11:09:27.226619 19284 consensus_queue.cc:260] T 5d9378f261e245e0a129ece0c12f32d0 P 26f75e5f673641c98de4d403c5f81f0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f14a29429c424983a63a24c90ce430" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46839 } } peers { permanent_uuid: "26f75e5f673641c98de4d403c5f81f0d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46653 } } peers { permanent_uuid: "0f6c507c4ea4437889dff506e520f7fd" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39643 } }
I20260812 11:09:27.226835 19282 consensus_queue.cc:260] T 5d9378f261e245e0a129ece0c12f32d0 P 0f6c507c4ea4437889dff506e520f7fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f14a29429c424983a63a24c90ce430" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46839 } } peers { permanent_uuid: "26f75e5f673641c98de4d403c5f81f0d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46653 } } peers { permanent_uuid: "0f6c507c4ea4437889dff506e520f7fd" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39643 } }
I20260812 11:09:27.227326 19045 heartbeater.cc:503] Master 127.17.44.254:42651 was elected leader, sending a full tablet report...
I20260812 11:09:27.227658 19284 ts_tablet_manager.cc:1428] T 5d9378f261e245e0a129ece0c12f32d0 P 26f75e5f673641c98de4d403c5f81f0d: Time spent starting tablet: real 0.011s user 0.004s sys 0.006s
I20260812 11:09:27.228755 19280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:27.224001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:37413 (local address 127.17.44.193:39643)
0812 11:09:27.224343 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:27.224354 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:27.224372 (+ 18us) server_negotiation.cc:407] Connection header received
0812 11:09:27.224429 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:27.224431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:27.224479 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:27.224566 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:09:27.225268 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.225961 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.227518 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.228122 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.228257 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:27.228316 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:27.228572 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 11:09:27.228642 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260812 11:09:27.228758 19251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:27.222991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:39643 (local address 127.17.44.195:37413)
0812 11:09:27.223545 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:27.223560 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:27.223661 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:27.224693 (+ 1032us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:27.224696 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:27.224713 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:09:27.224924 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:27.224933 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.226084 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.226091 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:27.227382 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:27.227414 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.228256 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.228260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:27.228440 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:27.228459 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 11:09:27.228555 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 11:09:27.228626 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":3,"thread_start_us":138,"threads_started":1}
I20260812 11:09:27.229024 19282 ts_tablet_manager.cc:1428] T 5d9378f261e245e0a129ece0c12f32d0 P 0f6c507c4ea4437889dff506e520f7fd: Time spent starting tablet: real 0.012s user 0.009s sys 0.003s
I20260812 11:09:27.229357 18821 heartbeater.cc:503] Master 127.17.44.254:42651 was elected leader, sending a full tablet report...
I20260812 11:09:27.229493 18747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d9378f261e245e0a129ece0c12f32d0" candidate_uuid: "c4f14a29429c424983a63a24c90ce430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f6c507c4ea4437889dff506e520f7fd" is_pre_election: true
I20260812 11:09:27.229839 18747 raft_consensus.cc:2454] T 5d9378f261e245e0a129ece0c12f32d0 P 0f6c507c4ea4437889dff506e520f7fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4f14a29429c424983a63a24c90ce430 in term 0.
I20260812 11:09:27.230439 19090 leader_election.cc:302] T 5d9378f261e245e0a129ece0c12f32d0 P c4f14a29429c424983a63a24c90ce430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f6c507c4ea4437889dff506e520f7fd, c4f14a29429c424983a63a24c90ce430; no voters:
I20260812 11:09:27.230690 19289 raft_consensus.cc:2781] T 5d9378f261e245e0a129ece0c12f32d0 P c4f14a29429c424983a63a24c90ce430 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:09:27.230887 19289 raft_consensus.cc:482] T 5d9378f261e245e0a129ece0c12f32d0 P c4f14a29429c424983a63a24c90ce430 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:09:27.231040 19289 raft_consensus.cc:3037] T 5d9378f261e245e0a129ece0c12f32d0 P c4f14a29429c424983a63a24c90ce430 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:27.233261 19289 raft_consensus.cc:504] T 5d9378f261e245e0a129ece0c12f32d0 P c4f14a29429c424983a63a24c90ce430 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f14a29429c424983a63a24c90ce430" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46839 } } peers { permanent_uuid: "26f75e5f673641c98de4d403c5f81f0d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46653 } } peers { permanent_uuid: "0f6c507c4ea4437889dff506e520f7fd" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39643 } }
I20260812 11:09:27.233788 19289 leader_election.cc:288] T 5d9378f261e245e0a129ece0c12f32d0 P c4f14a29429c424983a63a24c90ce430 [CANDIDATE]: Term 1 election: Requested vote from peers 26f75e5f673641c98de4d403c5f81f0d (127.17.44.194:46653), 0f6c507c4ea4437889dff506e520f7fd (127.17.44.193:39643)
I20260812 11:09:27.234184 18747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d9378f261e245e0a129ece0c12f32d0" candidate_uuid: "c4f14a29429c424983a63a24c90ce430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f6c507c4ea4437889dff506e520f7fd"
I20260812 11:09:27.234301 18747 raft_consensus.cc:3037] T 5d9378f261e245e0a129ece0c12f32d0 P 0f6c507c4ea4437889dff506e520f7fd [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:27.234376 19276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:27.224028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:46253 (local address 127.17.44.194:46653)
0812 11:09:27.224140 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:27.224145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:27.224160 (+ 15us) server_negotiation.cc:407] Connection header received
0812 11:09:27.224209 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:27.224213 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:27.224258 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:27.224346 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:09:27.230153 (+ 5807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.230836 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.232768 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.233492 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.233615 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:27.233663 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:27.234164 (+ 501us) server_negotiation.cc:299] Negotiation successful
0812 11:09:27.234255 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 11:09:27.234470 19290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:27.222646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:46653 (local address 127.17.44.195:46253)
0812 11:09:27.223398 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:27.223415 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:27.223584 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:27.228229 (+ 4645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:27.228232 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:27.228255 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:09:27.230002 (+ 1747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:27.230013 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.231012 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.231015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:27.232625 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:27.232635 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.233824 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.233827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:27.234006 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:27.234027 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 11:09:27.234129 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 11:09:27.234201 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":1029}
I20260812 11:09:27.235312 18965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d9378f261e245e0a129ece0c12f32d0" candidate_uuid: "c4f14a29429c424983a63a24c90ce430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26f75e5f673641c98de4d403c5f81f0d" is_pre_election: true
I20260812 11:09:27.235669 18965 raft_consensus.cc:2454] T 5d9378f261e245e0a129ece0c12f32d0 P 26f75e5f673641c98de4d403c5f81f0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4f14a29429c424983a63a24c90ce430 in term 0.
I20260812 11:09:27.235669 18963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d9378f261e245e0a129ece0c12f32d0" candidate_uuid: "c4f14a29429c424983a63a24c90ce430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26f75e5f673641c98de4d403c5f81f0d"
I20260812 11:09:27.235845 18963 raft_consensus.cc:3037] T 5d9378f261e245e0a129ece0c12f32d0 P 26f75e5f673641c98de4d403c5f81f0d [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:27.236445 18747 raft_consensus.cc:2454] T 5d9378f261e245e0a129ece0c12f32d0 P 0f6c507c4ea4437889dff506e520f7fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4f14a29429c424983a63a24c90ce430 in term 1.
I20260812 11:09:27.236774 19090 leader_election.cc:302] T 5d9378f261e245e0a129ece0c12f32d0 P c4f14a29429c424983a63a24c90ce430 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f6c507c4ea4437889dff506e520f7fd, c4f14a29429c424983a63a24c90ce430; no voters:
I20260812 11:09:27.237035 19289 raft_consensus.cc:2781] T 5d9378f261e245e0a129ece0c12f32d0 P c4f14a29429c424983a63a24c90ce430 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:27.237229 19289 raft_consensus.cc:686] T 5d9378f261e245e0a129ece0c12f32d0 P c4f14a29429c424983a63a24c90ce430 [term 1 LEADER]: Becoming Leader. State: Replica: c4f14a29429c424983a63a24c90ce430, State: Running, Role: LEADER
I20260812 11:09:27.237520 19289 consensus_queue.cc:237] T 5d9378f261e245e0a129ece0c12f32d0 P c4f14a29429c424983a63a24c90ce430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f14a29429c424983a63a24c90ce430" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46839 } } peers { permanent_uuid: "26f75e5f673641c98de4d403c5f81f0d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46653 } } peers { permanent_uuid: "0f6c507c4ea4437889dff506e520f7fd" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39643 } }
I20260812 11:09:27.237803 18963 raft_consensus.cc:2454] T 5d9378f261e245e0a129ece0c12f32d0 P 26f75e5f673641c98de4d403c5f81f0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4f14a29429c424983a63a24c90ce430 in term 1.
I20260812 11:09:27.241977 18534 catalog_manager.cc:5495] T 5d9378f261e245e0a129ece0c12f32d0 P c4f14a29429c424983a63a24c90ce430 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4f14a29429c424983a63a24c90ce430 (127.17.44.195). New cstate: current_term: 1 leader_uuid: "c4f14a29429c424983a63a24c90ce430" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4f14a29429c424983a63a24c90ce430" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "26f75e5f673641c98de4d403c5f81f0d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f6c507c4ea4437889dff506e520f7fd" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39643 } health_report { overall_health: UNKNOWN } } }
Aug 12 11:09:27 dist-test-slave-drl0 krb5kdc[18458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.193@KRBTEST.COM
I20260812 11:09:27.263598 19280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:27.252932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55512 (local address 127.17.44.193:39643)
0812 11:09:27.253133 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:27.253143 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:27.253400 (+ 257us) server_negotiation.cc:407] Connection header received
0812 11:09:27.253478 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:27.253481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:27.253527 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:27.253597 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:27.254408 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.254901 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.256205 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.256414 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.256496 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:27.259398 (+ 2902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:27.259408 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:27.259422 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:27.259446 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:27.261666 (+ 2220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:27.262132 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:27.262138 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:27.262139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:27.262219 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:27.262530 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:27.262533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:27.262534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:27.262788 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:27.263025 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:27.263276 (+ 251us) server_negotiation.cc:299] Negotiation successful
0812 11:09:27.263428 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Aug 12 11:09:27 dist-test-slave-drl0 krb5kdc[18458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532967, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 11:09:27 dist-test-slave-drl0 krb5kdc[18458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532967, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.44.193@KRBTEST.COM
I20260812 11:09:27.298730 19280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:27.287255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55526 (local address 127.17.44.193:39643)
0812 11:09:27.287414 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:27.287419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:27.287778 (+ 359us) server_negotiation.cc:407] Connection header received
0812 11:09:27.288294 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:27.288329 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:27.288389 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:27.288461 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:27.289354 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.289884 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.291338 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.291566 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.291656 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:27.295162 (+ 3506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:27.295166 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:27.295168 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:27.295191 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:27.296705 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:27.297146 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:27.297149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:27.297150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:27.297234 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:27.297784 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:27.297787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:27.297788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:27.297961 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:27.298079 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:27.298396 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 11:09:27.298495 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 11:09:27 dist-test-slave-drl0 krb5kdc[18458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 11:09:27 dist-test-slave-drl0 krb5kdc[18458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.193@KRBTEST.COM
I20260812 11:09:27.330660 19280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:27.320440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55532 (local address 127.17.44.193:39643)
0812 11:09:27.320657 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:27.320663 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:27.320905 (+ 242us) server_negotiation.cc:407] Connection header received
0812 11:09:27.321010 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:27.321013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:27.321061 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:27.321136 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:27.321918 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.322652 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.323863 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:27.324108 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:27.324207 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:27.327034 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:27.327037 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:27.327039 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:27.327061 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:27.328811 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:27.329280 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:27.329283 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:27.329284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:27.329381 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:27.329713 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:27.329717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:27.329718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:27.329967 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:27.330107 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:27.330402 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 11:09:27.330535 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260812 11:09:27.332249 18691 tablet_service.cc:2908] Rejecting scan request for tablet 5d9378f261e245e0a129ece0c12f32d0: Uninitialized: clean time has not yet been initialized
I20260812 11:09:27.334553 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 18627
I20260812 11:09:27.350520 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 18832
I20260812 11:09:27.363221 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 19059
I20260812 11:09:27.377277 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 18480
2026-08-12T11:09:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2721 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:09:27 dist-test-slave-drl0 krb5kdc[19316](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:09:27 dist-test-slave-drl0 krb5kdc[19316](info): set up 2 sockets
Aug 12 11:09:27 dist-test-slave-drl0 krb5kdc[19316](info): commencing operation
krb5kdc: starting...
W20260812 11:09:28.662729 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:09:28 dist-test-slave-drl0 krb5kdc[19316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:09:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:09:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:09:28.799338 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:42883
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:40889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:42883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:28.925437 19335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:28.925688 19335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:28.925736 19335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:28.932574 19335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:09:28.932649 19335 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 11:09:28.932677 19335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:28.932703 19335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:28.932729 19335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:09:28.932751 19335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:09:28.940412 19335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:40889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:42883
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:42883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:28.941864 19335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:28.943100 19335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:28.953415 19342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:28.953441 19341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:28.953430 19345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:28.953698 19335 server_base.cc:1034] running on GCE node
I20260812 11:09:28.954514 19335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:28.956023 19335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:28.957237 19335 hybrid_clock.cc:648] HybridClock initialized: now 1786532968957216 us; error 44 us; skew 500 ppm
Aug 12 11:09:28 dist-test-slave-drl0 krb5kdc[19316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532968, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:28.960320 19335 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:09:28.961309 19335 webserver.cc:460] Webserver started at http://127.17.44.254:40519/ using document root <none> and password file <none>
I20260812 11:09:28.961884 19335 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:28.961949 19335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:28.962170 19335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:28.964466 19335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "c50ee1e817b74c8ab9ad1a0f5594bcda"
format_stamp: "Formatted at 2026-08-12 11:09:28 on dist-test-slave-drl0"
server_key: "ed92afe077be8a86da5d52a4eb742d9e"
server_key_iv: "df2f292bcffb1a7a9e2b1148917242a4"
server_key_version: "encryptionkey@0"
I20260812 11:09:28.965070 19335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "c50ee1e817b74c8ab9ad1a0f5594bcda"
format_stamp: "Formatted at 2026-08-12 11:09:28 on dist-test-slave-drl0"
server_key: "ed92afe077be8a86da5d52a4eb742d9e"
server_key_iv: "df2f292bcffb1a7a9e2b1148917242a4"
server_key_version: "encryptionkey@0"
I20260812 11:09:28.969288 19335 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 11:09:28.973198 19357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:28.975040 19335 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 11:09:28.975272 19335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "c50ee1e817b74c8ab9ad1a0f5594bcda"
format_stamp: "Formatted at 2026-08-12 11:09:28 on dist-test-slave-drl0"
server_key: "ed92afe077be8a86da5d52a4eb742d9e"
server_key_iv: "df2f292bcffb1a7a9e2b1148917242a4"
server_key_version: "encryptionkey@0"
I20260812 11:09:28.975440 19335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:28.995154 19335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:28.999336 19335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:28.999570 19335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:29.008558 19335 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:42883
I20260812 11:09:29.008628 19446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:42883 every 8 connection(s)
I20260812 11:09:29.009545 19335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:09:29.012446 19447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:29.018460 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 19335
I20260812 11:09:29.018635 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:09:29.018991 17587 external_mini_cluster.cc:1428] Setting key c7b885ca5d94a0acf077788ec15e07b4
I20260812 11:09:29.019668 19447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda: Bootstrap starting.
I20260812 11:09:29.022253 19447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:29.023066 19447 log.cc:824] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:29.025079 19447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda: No bootstrap required, opened a new log
Aug 12 11:09:29 dist-test-slave-drl0 krb5kdc[19316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:29.029313 19447 raft_consensus.cc:348] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c50ee1e817b74c8ab9ad1a0f5594bcda" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42883 } }
I20260812 11:09:29.029561 19447 raft_consensus.cc:374] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:29.029635 19447 raft_consensus.cc:729] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c50ee1e817b74c8ab9ad1a0f5594bcda, State: Initialized, Role: FOLLOWER
I20260812 11:09:29.030122 19447 consensus_queue.cc:260] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c50ee1e817b74c8ab9ad1a0f5594bcda" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42883 } }
I20260812 11:09:29.030282 19447 raft_consensus.cc:388] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:09:29.030372 19447 raft_consensus.cc:482] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:09:29.030555 19447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:29.031889 19447 raft_consensus.cc:504] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c50ee1e817b74c8ab9ad1a0f5594bcda" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42883 } }
I20260812 11:09:29.032279 19447 leader_election.cc:302] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c50ee1e817b74c8ab9ad1a0f5594bcda; no voters:
I20260812 11:09:29.032651 19447 leader_election.cc:288] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:09:29.032800 19459 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:29.033036 19459 raft_consensus.cc:686] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda [term 1 LEADER]: Becoming Leader. State: Replica: c50ee1e817b74c8ab9ad1a0f5594bcda, State: Running, Role: LEADER
I20260812 11:09:29.033380 19459 consensus_queue.cc:237] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c50ee1e817b74c8ab9ad1a0f5594bcda" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42883 } }
I20260812 11:09:29.033952 19447 sys_catalog.cc:564] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:09:29.035007 19461 sys_catalog.cc:455] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda [sys.catalog]: SysCatalogTable state changed. Reason: New leader c50ee1e817b74c8ab9ad1a0f5594bcda. Latest consensus state: current_term: 1 leader_uuid: "c50ee1e817b74c8ab9ad1a0f5594bcda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c50ee1e817b74c8ab9ad1a0f5594bcda" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42883 } } }
I20260812 11:09:29.035168 19461 sys_catalog.cc:458] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:29.035440 19462 sys_catalog.cc:455] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c50ee1e817b74c8ab9ad1a0f5594bcda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c50ee1e817b74c8ab9ad1a0f5594bcda" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42883 } } }
I20260812 11:09:29.035573 19462 sys_catalog.cc:458] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:29.036130 19471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:09:29.037242 19456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:29.020713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60412 (local address 127.17.44.254:42883)
0812 11:09:29.021193 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:29.021208 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:29.021243 (+ 35us) server_negotiation.cc:407] Connection header received
0812 11:09:29.021939 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:29.021956 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:29.022271 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:29.022750 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:29.023604 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.024581 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.025597 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.026076 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.026278 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:29.030013 (+ 3735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:29.030018 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:29.030030 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:29.030059 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:29.032285 (+ 2226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:29.033038 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:29.033046 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:29.033055 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:29.033198 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:29.033552 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:29.033555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:29.033557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:29.034076 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:29.034276 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:29.034780 (+ 504us) server_negotiation.cc:299] Negotiation successful
0812 11:09:29.036073 (+ 1293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":166,"threads_started":1}
I20260812 11:09:29.039350 19471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:09:29.044863 19471 catalog_manager.cc:1347] Generated new cluster ID: 033e99e4f5ed4cffa503fa84d0dba29c
I20260812 11:09:29.044926 19471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:09:29.055810 19471 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:09:29.057091 19471 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:09:29.075662 19471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda: Generated new TSK 0
I20260812 11:09:29.076481 19471 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
I20260812 11:09:29.115324 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:42883
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:40889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:29.243115 19498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:29.243366 19498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:29.243417 19498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:29.247996 19498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:29.248116 19498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:29.248252 19498 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 11:09:29.248358 19498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:09:29.256268 19498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:40889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.44.254:42883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:29.257277 19498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:29.258383 19498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:29.268512 19503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:29.268548 19504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:29.268697 19498 server_base.cc:1034] running on GCE node
W20260812 11:09:29.268502 19508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:29.269279 19498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:29.269894 19498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:29.271039 19498 hybrid_clock.cc:648] HybridClock initialized: now 1786532969271027 us; error 33 us; skew 500 ppm
Aug 12 11:09:29 dist-test-slave-drl0 krb5kdc[19316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532969, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:29.274113 19498 init.cc:377] Logged in from keytab as kudu/127.17.44.193@KRBTEST.COM (short username kudu)
I20260812 11:09:29.275104 19498 webserver.cc:460] Webserver started at http://127.17.44.193:36039/ using document root <none> and password file <none>
I20260812 11:09:29.275684 19498 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:29.275763 19498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:29.275986 19498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:29.278049 19498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "f41759f922954da9b6d51cbfe3691a98"
format_stamp: "Formatted at 2026-08-12 11:09:29 on dist-test-slave-drl0"
server_key: "6ce892aa64903541e979ffa35dd20563"
server_key_iv: "b2ddd594bc2b5679e9a456ec5d9ef620"
server_key_version: "encryptionkey@0"
I20260812 11:09:29.278630 19498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "f41759f922954da9b6d51cbfe3691a98"
format_stamp: "Formatted at 2026-08-12 11:09:29 on dist-test-slave-drl0"
server_key: "6ce892aa64903541e979ffa35dd20563"
server_key_iv: "b2ddd594bc2b5679e9a456ec5d9ef620"
server_key_version: "encryptionkey@0"
I20260812 11:09:29.283149 19498 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260812 11:09:29.286058 19516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:29.287786 19498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:09:29.287931 19498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "f41759f922954da9b6d51cbfe3691a98"
format_stamp: "Formatted at 2026-08-12 11:09:29 on dist-test-slave-drl0"
server_key: "6ce892aa64903541e979ffa35dd20563"
server_key_iv: "b2ddd594bc2b5679e9a456ec5d9ef620"
server_key_version: "encryptionkey@0"
I20260812 11:09:29.288045 19498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:29.300661 19498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:29.303980 19498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:29.304204 19498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:29.305644 19498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:29.305702 19498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:29.305780 19498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:29.305814 19498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 11:09:29 dist-test-slave-drl0 krb5kdc[19316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532969, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:29.318037 19498 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:38151
I20260812 11:09:29.318125 19701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:38151 every 8 connection(s)
I20260812 11:09:29.319252 19498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:09:29.322602 19456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:29.306878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:43783 (local address 127.17.44.254:42883)
0812 11:09:29.307203 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:29.307208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:29.308627 (+ 1419us) server_negotiation.cc:407] Connection header received
0812 11:09:29.310182 (+ 1555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:29.310186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:29.310253 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:29.310362 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:29.312488 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.313225 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.314736 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.315065 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.315193 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:29.318063 (+ 2870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:29.318069 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:29.318077 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:29.318123 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:29.319660 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:29.320273 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:29.320277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:29.320279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:29.320369 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:29.320838 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:29.320843 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:29.320844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:29.321208 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:29.321353 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:29.322002 (+ 649us) server_negotiation.cc:299] Negotiation successful
0812 11:09:29.322392 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260812 11:09:29.322878 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 19498
I20260812 11:09:29.323011 19532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:29.307259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42883 (local address 127.17.44.193:43783)
0812 11:09:29.308285 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:29.308413 (+ 128us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:29.309892 (+ 1479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:29.310839 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:29.310855 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:29.311422 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:29.312225 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:29.312251 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.313400 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.313404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:29.314556 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:29.314574 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.315240 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.315245 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:29.315436 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:29.316234 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:29.316255 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:29.317887 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:29.319821 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:29.319826 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:29.319840 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:29.320165 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:29.320497 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:29.320499 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:29.320501 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:29.320643 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:29.321350 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:29.321355 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:29.321674 (+ 319us) client_negotiation.cc:769] Sending connection context
0812 11:09:29.321858 (+ 184us) client_negotiation.cc:240] Negotiation successful
0812 11:09:29.322113 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":163,"threads_started":1}
I20260812 11:09:29.323038 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:09:29.323361 17587 external_mini_cluster.cc:1428] Setting key 46c2b8804eba1f6bc353d58977f82f49
I20260812 11:09:29.325047 19703 heartbeater.cc:347] Connected to a master server at 127.17.44.254:42883
I20260812 11:09:29.325362 19703 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:29.326223 19703 heartbeater.cc:511] Master 127.17.44.254:42883 requested a full tablet report, sending...
I20260812 11:09:29.329188 19375 ts_manager.cc:194] Registered new tserver with Master: f41759f922954da9b6d51cbfe3691a98 (127.17.44.193:38151)
I20260812 11:09:29.331278 19375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.193@KRBTEST.COM'} at 127.17.44.193:43783
I20260812 11:09:29.336124 19456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:29.327216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:56361 (local address 127.17.44.254:42883)
0812 11:09:29.327352 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:29.327357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:29.327896 (+ 539us) server_negotiation.cc:407] Connection header received
0812 11:09:29.328052 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:29.328058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:29.328115 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:29.328209 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:29.328779 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.329314 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.330213 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.330490 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.330603 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:29.331828 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:29.331832 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:29.331834 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:29.331858 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:29.333848 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:29.334414 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:29.334416 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:29.334418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:29.334493 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:29.334918 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:29.334922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:29.334923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:29.335244 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:29.335368 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:29.335819 (+ 451us) server_negotiation.cc:299] Negotiation successful
0812 11:09:29.335931 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":54}
I20260812 11:09:29.336189 19714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:29.327158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42883 (local address 127.17.44.193:56361)
0812 11:09:29.327663 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:29.327678 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:29.327937 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:29.328239 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:29.328243 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:29.328446 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:29.328549 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:29.328627 (+ 78us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.329506 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.329510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:29.330084 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:29.330093 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.330610 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.330614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:29.330713 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:29.331270 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:29.331282 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:29.331692 (+ 410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:29.333987 (+ 2295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:29.333994 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:29.333997 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:29.334293 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:29.334601 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:29.334606 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:29.334610 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:29.334779 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:29.335384 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:29.335387 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:29.335565 (+ 178us) client_negotiation.cc:769] Sending connection context
0812 11:09:29.335756 (+ 191us) client_negotiation.cc:240] Negotiation successful
0812 11:09:29.335911 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":137,"threads_started":1}
WARNING: no policy specified for kudu/127.17.44.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
I20260812 11:09:29.358547 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:42883
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:40889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:29.486800 19717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:29.487090 19717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:29.487136 19717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:29.491053 19717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:29.491106 19717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:29.491143 19717 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 11:09:29.491174 19717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:09:29.497102 19717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:40889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.44.254:42883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:29.498095 19717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:29.499130 19717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:29.508937 19725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:29.508952 19726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:29.508947 19728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:29.509100 19717 server_base.cc:1034] running on GCE node
I20260812 11:09:29.509685 19717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:29.510368 19717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:29.511552 19717 hybrid_clock.cc:648] HybridClock initialized: now 1786532969511491 us; error 74 us; skew 500 ppm
Aug 12 11:09:29 dist-test-slave-drl0 krb5kdc[19316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532969, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:29.514571 19717 init.cc:377] Logged in from keytab as kudu/127.17.44.194@KRBTEST.COM (short username kudu)
I20260812 11:09:29.515568 19717 webserver.cc:460] Webserver started at http://127.17.44.194:39911/ using document root <none> and password file <none>
I20260812 11:09:29.516152 19717 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:29.516214 19717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:29.516434 19717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:29.518828 19717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "936988d8930a425ab67afb31c00c7538"
format_stamp: "Formatted at 2026-08-12 11:09:29 on dist-test-slave-drl0"
server_key: "f081b2f4fbf9f99f7d97c883bc457516"
server_key_iv: "9e71560a7c63c2d3717f7a5eaca7878b"
server_key_version: "encryptionkey@0"
I20260812 11:09:29.519491 19717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "936988d8930a425ab67afb31c00c7538"
format_stamp: "Formatted at 2026-08-12 11:09:29 on dist-test-slave-drl0"
server_key: "f081b2f4fbf9f99f7d97c883bc457516"
server_key_iv: "9e71560a7c63c2d3717f7a5eaca7878b"
server_key_version: "encryptionkey@0"
I20260812 11:09:29.523607 19717 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 11:09:29.526206 19740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:29.527814 19717 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 11:09:29.527951 19717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "936988d8930a425ab67afb31c00c7538"
format_stamp: "Formatted at 2026-08-12 11:09:29 on dist-test-slave-drl0"
server_key: "f081b2f4fbf9f99f7d97c883bc457516"
server_key_iv: "9e71560a7c63c2d3717f7a5eaca7878b"
server_key_version: "encryptionkey@0"
I20260812 11:09:29.528060 19717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:29.542384 19717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:29.546828 19717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:29.547149 19717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:29.548717 19717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:29.548799 19717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:29.548871 19717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:29.548924 19717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 11:09:29 dist-test-slave-drl0 krb5kdc[19316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532969, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:29.561319 19717 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:41553
I20260812 11:09:29.561481 19920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:41553 every 8 connection(s)
I20260812 11:09:29.562697 19717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:09:29.564934 19456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:29.550558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:46959 (local address 127.17.44.254:42883)
0812 11:09:29.550692 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:29.550696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:29.552079 (+ 1383us) server_negotiation.cc:407] Connection header received
0812 11:09:29.553553 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:29.553556 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:29.553617 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:29.553728 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:29.555471 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.556394 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.557561 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.557801 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.557926 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:29.560718 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:29.560724 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:29.560730 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:29.560763 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:29.562506 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:29.563139 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:29.563142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:29.563144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:29.563232 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:29.563656 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:29.563661 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:29.563663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:29.563893 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:29.564009 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:29.564639 (+ 630us) server_negotiation.cc:299] Negotiation successful
0812 11:09:29.564759 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:09:29.565697 19749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:29.551018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42883 (local address 127.17.44.194:46959)
0812 11:09:29.551780 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:29.551856 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:29.553307 (+ 1451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:29.553910 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:29.553919 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:29.554497 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:29.555272 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:29.555291 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.556543 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.556547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:29.557423 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:29.557433 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.557968 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.557972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:29.558156 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:29.558763 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:29.558782 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:29.560486 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:29.562627 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:29.562633 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:29.562648 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:29.563019 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:29.563334 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:29.563337 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:29.563339 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:29.563536 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:29.564177 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:29.564181 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:29.564393 (+ 212us) client_negotiation.cc:769] Sending connection context
0812 11:09:29.564575 (+ 182us) client_negotiation.cc:240] Negotiation successful
0812 11:09:29.564895 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":134,"threads_started":1}
I20260812 11:09:29.566412 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 19717
I20260812 11:09:29.566651 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:09:29.567307 19921 heartbeater.cc:347] Connected to a master server at 127.17.44.254:42883
I20260812 11:09:29.567484 17587 external_mini_cluster.cc:1428] Setting key daab98ded1d3d3b557bde2a9966f5f3c
I20260812 11:09:29.567528 19921 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:29.568138 19921 heartbeater.cc:511] Master 127.17.44.254:42883 requested a full tablet report, sending...
I20260812 11:09:29.569249 19375 ts_manager.cc:194] Registered new tserver with Master: 936988d8930a425ab67afb31c00c7538 (127.17.44.194:41553)
I20260812 11:09:29.570001 19375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.194@KRBTEST.COM'} at 127.17.44.194:46959
I20260812 11:09:29.578536 19456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:29.570305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:40065 (local address 127.17.44.254:42883)
0812 11:09:29.570415 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:29.570418 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:29.570839 (+ 421us) server_negotiation.cc:407] Connection header received
0812 11:09:29.570970 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:29.570975 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:29.571017 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:29.571087 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:29.571735 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.572541 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.573650 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.573884 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.573997 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:29.575241 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:29.575245 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:29.575248 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:29.575271 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:29.576831 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:29.577278 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:29.577281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:29.577283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:29.577372 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:29.577671 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:29.577674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:29.577676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:29.577892 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:29.578008 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:29.578269 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 11:09:29.578372 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 11:09:29.578548 19938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:29.570442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42883 (local address 127.17.44.194:40065)
0812 11:09:29.570734 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:29.570747 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:29.570872 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:29.571195 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:29.571198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:29.571407 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:29.571586 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:29.571596 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.572717 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.572722 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:29.573519 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:29.573529 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.574003 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.574006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:29.574101 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:29.574624 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:29.574638 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:29.575112 (+ 474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:29.576954 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:29.576959 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:29.576961 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:29.577164 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:29.577477 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:29.577482 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:29.577484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:29.577572 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:29.577998 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:29.578001 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:29.578104 (+ 103us) client_negotiation.cc:769] Sending connection context
0812 11:09:29.578215 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 11:09:29.578366 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":114,"threads_started":1}
WARNING: no policy specified for kudu/127.17.44.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
I20260812 11:09:29.598376 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:42883
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:40889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:29.731272 19944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:29.731743 19944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:29.731833 19944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:29.738171 19944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:29.738371 19944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:29.738466 19944 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 11:09:29.738566 19944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:09:29.745771 19944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:40889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.44.254:42883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:29.747072 19944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:29.748188 19944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:29.758348 19952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:29.758435 19955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:29.758348 19953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:29.758569 19944 server_base.cc:1034] running on GCE node
I20260812 11:09:29.759135 19944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:29.759721 19944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:29.761003 19944 hybrid_clock.cc:648] HybridClock initialized: now 1786532969760973 us; error 80 us; skew 500 ppm
Aug 12 11:09:29 dist-test-slave-drl0 krb5kdc[19316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532969, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:29.764684 19944 init.cc:377] Logged in from keytab as kudu/127.17.44.195@KRBTEST.COM (short username kudu)
I20260812 11:09:29.765674 19944 webserver.cc:460] Webserver started at http://127.17.44.195:44637/ using document root <none> and password file <none>
I20260812 11:09:29.766285 19944 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:29.766389 19944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:29.766633 19944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:29.769932 19944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "77e67f84942f4e58a3990fed8e009e85"
format_stamp: "Formatted at 2026-08-12 11:09:29 on dist-test-slave-drl0"
server_key: "44be50aca9066f295c59fb60a7dc99b5"
server_key_iv: "28f8e96a1210195167459f79469b4292"
server_key_version: "encryptionkey@0"
I20260812 11:09:29.770515 19944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "77e67f84942f4e58a3990fed8e009e85"
format_stamp: "Formatted at 2026-08-12 11:09:29 on dist-test-slave-drl0"
server_key: "44be50aca9066f295c59fb60a7dc99b5"
server_key_iv: "28f8e96a1210195167459f79469b4292"
server_key_version: "encryptionkey@0"
I20260812 11:09:29.775583 19944 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260812 11:09:29.779220 19969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:29.780750 19944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:09:29.780896 19944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "77e67f84942f4e58a3990fed8e009e85"
format_stamp: "Formatted at 2026-08-12 11:09:29 on dist-test-slave-drl0"
server_key: "44be50aca9066f295c59fb60a7dc99b5"
server_key_iv: "28f8e96a1210195167459f79469b4292"
server_key_version: "encryptionkey@0"
I20260812 11:09:29.781032 19944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:29.804814 19944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:29.808068 19944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:29.808293 19944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:29.809900 19944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:29.809991 19944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:29.810073 19944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:29.810118 19944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 11:09:29 dist-test-slave-drl0 krb5kdc[19316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532969, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:29.823937 19944 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:45919
I20260812 11:09:29.824009 20147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:45919 every 8 connection(s)
I20260812 11:09:29.824852 19944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:09:29.826557 19456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:29.811693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:38315 (local address 127.17.44.254:42883)
0812 11:09:29.811852 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:29.811855 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:29.813107 (+ 1252us) server_negotiation.cc:407] Connection header received
0812 11:09:29.814405 (+ 1298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:29.814409 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:29.814477 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:29.814598 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:29.816148 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.817064 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.818739 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.819094 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.819226 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:29.822451 (+ 3225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:29.822455 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:29.822458 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:29.822491 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:29.824317 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:29.824841 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:29.824846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:29.824848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:29.824957 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:29.825416 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:29.825420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:29.825422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:29.825665 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:29.825768 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:29.826265 (+ 497us) server_negotiation.cc:299] Negotiation successful
0812 11:09:29.826433 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 11:09:29.827385 19979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:29.812383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42883 (local address 127.17.44.195:38315)
0812 11:09:29.812946 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:29.812990 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:29.814149 (+ 1159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:29.814793 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:29.814804 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:29.815340 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:29.815837 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:29.815851 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.817219 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.817225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:29.818518 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:29.818530 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.819321 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.819329 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:29.819570 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:29.820273 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:29.820301 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:29.822270 (+ 1969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:29.824444 (+ 2174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:29.824450 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:29.824464 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:29.824732 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:29.825064 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:29.825067 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:29.825069 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:29.825277 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:29.825762 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:29.825769 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:29.826027 (+ 258us) client_negotiation.cc:769] Sending connection context
0812 11:09:29.826211 (+ 184us) client_negotiation.cc:240] Negotiation successful
0812 11:09:29.826508 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":154,"threads_started":1}
I20260812 11:09:29.828646 20148 heartbeater.cc:347] Connected to a master server at 127.17.44.254:42883
I20260812 11:09:29.828866 20148 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:29.829459 20148 heartbeater.cc:511] Master 127.17.44.254:42883 requested a full tablet report, sending...
I20260812 11:09:29.829566 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 19944
I20260812 11:09:29.829696 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:09:29.829994 17587 external_mini_cluster.cc:1428] Setting key 6e947a86832c45037673d14a8df6b39f
I20260812 11:09:29.830686 19379 ts_manager.cc:194] Registered new tserver with Master: 77e67f84942f4e58a3990fed8e009e85 (127.17.44.195:45919)
I20260812 11:09:29.831516 19379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.195@KRBTEST.COM'} at 127.17.44.195:38315
I20260812 11:09:29.832912 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:09:29.839669 19456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:29.831275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:52617 (local address 127.17.44.254:42883)
0812 11:09:29.831473 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:29.831477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:29.831667 (+ 190us) server_negotiation.cc:407] Connection header received
0812 11:09:29.831840 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:29.831843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:29.831884 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:29.831955 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:29.832564 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.833278 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.834364 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.834759 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.834906 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:29.836261 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:29.836265 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:29.836268 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:29.836291 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:29.837868 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:29.838328 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:29.838332 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:29.838334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:29.838446 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:29.838754 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:29.838758 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:29.838759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:29.838987 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:29.839089 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:29.839392 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 11:09:29.839525 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260812 11:09:29.839733 20155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:29.831233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42883 (local address 127.17.44.195:52617)
0812 11:09:29.831581 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:29.831600 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:29.831736 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:29.832065 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:29.832068 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:29.832281 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:29.832423 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:29.832431 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.833456 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.833459 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:29.834247 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:29.834257 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.835100 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.835104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:29.835217 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:29.835780 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:29.835792 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:29.836114 (+ 322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:29.837991 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:29.837996 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:29.837998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:29.838205 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:29.838566 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:29.838569 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:29.838573 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:29.838655 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:29.839091 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:29.839094 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:29.839213 (+ 119us) client_negotiation.cc:769] Sending connection context
0812 11:09:29.839334 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 11:09:29.839496 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":130,"threads_started":1}
Aug 12 11:09:29 dist-test-slave-drl0 krb5kdc[19316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532969, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 11:09:29 dist-test-slave-drl0 krb5kdc[19316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532969, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:29.865619 19456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:29.854788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60424 (local address 127.17.44.254:42883)
0812 11:09:29.855097 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:29.855101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:29.855129 (+ 28us) server_negotiation.cc:407] Connection header received
0812 11:09:29.855201 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:29.855204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:29.855266 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:29.855384 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:29.856349 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.857074 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.858722 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.859054 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.859151 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:29.862059 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:29.862062 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:29.862064 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:29.862088 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:29.863520 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:29.864049 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:29.864052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:29.864053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:29.864126 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:29.864509 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:29.864511 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:29.864513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:29.864695 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:29.864800 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:29.865314 (+ 514us) server_negotiation.cc:299] Negotiation successful
0812 11:09:29.865416 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
I20260812 11:09:29.870253 19375 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60424:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:09:29.874698 19375 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:09:29.891670 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:29.886894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55338 (local address 127.17.44.195:45919)
0812 11:09:29.887392 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:29.887412 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:29.887433 (+ 21us) server_negotiation.cc:407] Connection header received
0812 11:09:29.887502 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:29.887506 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:29.887690 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:29.887844 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:09:29.888092 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.889163 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.890492 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.891075 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.891184 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:29.891351 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:29.891413 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 11:09:29.891507 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":205,"threads_started":1}
I20260812 11:09:29.891709 20181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:29.886707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:45919 (local address 127.0.0.1:55338)
0812 11:09:29.887046 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:29.887059 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:29.887133 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:29.887838 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:29.887841 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:29.887861 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:09:29.887970 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:29.887975 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.889301 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.889306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:29.890375 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:29.890382 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.891186 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.891189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:29.891293 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:29.891321 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 11:09:29.891388 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 11:09:29.891553 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":27,"thread_start_us":96,"threads_started":1}
I20260812 11:09:29.892000 20178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:29.885892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51518 (local address 127.17.44.193:38151)
0812 11:09:29.886360 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:29.886367 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:29.887008 (+ 641us) server_negotiation.cc:407] Connection header received
0812 11:09:29.887097 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:29.887103 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:29.887284 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:29.887427 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:09:29.887673 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.888792 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.890299 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.891095 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.891199 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:29.891708 (+ 509us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:29.891764 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 11:09:29.891865 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":82,"threads_started":1}
I20260812 11:09:29.892041 20172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:29.885352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50456 (local address 127.17.44.194:41553)
0812 11:09:29.885729 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:29.885747 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:29.886018 (+ 271us) server_negotiation.cc:407] Connection header received
0812 11:09:29.886240 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:29.886247 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:29.886491 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:29.887245 (+ 754us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:09:29.887400 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.888412 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.889979 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.890758 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.890868 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:29.891073 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:29.891807 (+ 734us) server_negotiation.cc:299] Negotiation successful
0812 11:09:29.891915 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":152,"threads_started":1}
I20260812 11:09:29.891721 20169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:29.885047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:41553 (local address 127.0.0.1:50456)
0812 11:09:29.885866 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:29.885914 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:29.886066 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:29.886819 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:29.886826 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:29.886854 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:09:29.887065 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:29.887077 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.888533 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.888538 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:29.889839 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:29.889850 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.891048 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.891052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:29.891231 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:29.891276 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 11:09:29.891392 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 11:09:29.891571 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"mutex_wait_us":136,"spinlock_wait_cycles":2944,"thread_start_us":93,"threads_started":1}
I20260812 11:09:29.892338 20171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:29.885194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:38151 (local address 127.0.0.1:51518)
0812 11:09:29.886760 (+ 1566us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:29.886790 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:29.886887 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:29.887425 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:29.887430 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:29.887452 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:09:29.887546 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:29.887552 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.888926 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.888929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:29.890165 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:29.890175 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:29.891361 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:29.891368 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:29.891455 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:29.891480 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 11:09:29.891541 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 11:09:29.892227 (+ 686us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1504,"thread_start_us":88,"threads_started":1}
I20260812 11:09:29.895097 19596 tablet_service.cc:1467] Processing CreateTablet for tablet 92ab3c15b2ae4cfe847a88cc4fab3a2a (DEFAULT_TABLE table=test-table [id=27f12243487549c7a71877654401ff1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:29.895203 19819 tablet_service.cc:1467] Processing CreateTablet for tablet 92ab3c15b2ae4cfe847a88cc4fab3a2a (DEFAULT_TABLE table=test-table [id=27f12243487549c7a71877654401ff1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:29.895200 20036 tablet_service.cc:1467] Processing CreateTablet for tablet 92ab3c15b2ae4cfe847a88cc4fab3a2a (DEFAULT_TABLE table=test-table [id=27f12243487549c7a71877654401ff1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:29.896180 19596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92ab3c15b2ae4cfe847a88cc4fab3a2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:29.896247 20036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92ab3c15b2ae4cfe847a88cc4fab3a2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:29.896240 19819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92ab3c15b2ae4cfe847a88cc4fab3a2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:29.902832 20183 tablet_bootstrap.cc:492] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P f41759f922954da9b6d51cbfe3691a98: Bootstrap starting.
I20260812 11:09:29.905942 20184 tablet_bootstrap.cc:492] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85: Bootstrap starting.
I20260812 11:09:29.907505 20183 tablet_bootstrap.cc:654] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P f41759f922954da9b6d51cbfe3691a98: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:29.908386 20186 tablet_bootstrap.cc:492] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 936988d8930a425ab67afb31c00c7538: Bootstrap starting.
I20260812 11:09:29.908834 20183 log.cc:824] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P f41759f922954da9b6d51cbfe3691a98: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:29.909242 20184 tablet_bootstrap.cc:654] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:29.910980 20184 log.cc:824] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:29.911752 20186 tablet_bootstrap.cc:654] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 936988d8930a425ab67afb31c00c7538: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:29.912043 20183 tablet_bootstrap.cc:492] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P f41759f922954da9b6d51cbfe3691a98: No bootstrap required, opened a new log
I20260812 11:09:29.912268 20183 ts_tablet_manager.cc:1397] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P f41759f922954da9b6d51cbfe3691a98: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260812 11:09:29.912613 20186 log.cc:824] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 936988d8930a425ab67afb31c00c7538: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:29.913219 20184 tablet_bootstrap.cc:492] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85: No bootstrap required, opened a new log
I20260812 11:09:29.913422 20184 ts_tablet_manager.cc:1397] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 11:09:29.914350 20186 tablet_bootstrap.cc:492] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 936988d8930a425ab67afb31c00c7538: No bootstrap required, opened a new log
I20260812 11:09:29.914564 20186 ts_tablet_manager.cc:1397] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 936988d8930a425ab67afb31c00c7538: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:09:29.917223 20183 raft_consensus.cc:348] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P f41759f922954da9b6d51cbfe3691a98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:29.917501 20183 raft_consensus.cc:374] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P f41759f922954da9b6d51cbfe3691a98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:29.917552 20183 raft_consensus.cc:729] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P f41759f922954da9b6d51cbfe3691a98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41759f922954da9b6d51cbfe3691a98, State: Initialized, Role: FOLLOWER
I20260812 11:09:29.917627 20184 raft_consensus.cc:348] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:29.917914 20184 raft_consensus.cc:374] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:29.917970 20184 raft_consensus.cc:729] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e67f84942f4e58a3990fed8e009e85, State: Initialized, Role: FOLLOWER
I20260812 11:09:29.917837 20186 raft_consensus.cc:348] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 936988d8930a425ab67afb31c00c7538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:29.918042 20183 consensus_queue.cc:260] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P f41759f922954da9b6d51cbfe3691a98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:29.918151 20186 raft_consensus.cc:374] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 936988d8930a425ab67afb31c00c7538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:29.918213 20186 raft_consensus.cc:729] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 936988d8930a425ab67afb31c00c7538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 936988d8930a425ab67afb31c00c7538, State: Initialized, Role: FOLLOWER
I20260812 11:09:29.918409 20184 consensus_queue.cc:260] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:29.918673 20186 consensus_queue.cc:260] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 936988d8930a425ab67afb31c00c7538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:29.918777 19703 heartbeater.cc:503] Master 127.17.44.254:42883 was elected leader, sending a full tablet report...
I20260812 11:09:29.919101 20183 ts_tablet_manager.cc:1428] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P f41759f922954da9b6d51cbfe3691a98: Time spent starting tablet: real 0.007s user 0.005s sys 0.003s
I20260812 11:09:29.919309 20184 ts_tablet_manager.cc:1428] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 11:09:29.919587 20186 ts_tablet_manager.cc:1428] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 936988d8930a425ab67afb31c00c7538: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260812 11:09:29.919610 20148 heartbeater.cc:503] Master 127.17.44.254:42883 was elected leader, sending a full tablet report...
I20260812 11:09:29.919898 19921 heartbeater.cc:503] Master 127.17.44.254:42883 was elected leader, sending a full tablet report...
I20260812 11:09:30.040319 20195 raft_consensus.cc:482] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:09:30.040611 20195 raft_consensus.cc:504] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:30.042012 20195 leader_election.cc:288] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 936988d8930a425ab67afb31c00c7538 (127.17.44.194:41553), f41759f922954da9b6d51cbfe3691a98 (127.17.44.193:38151)
I20260812 11:09:30.045766 19979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:30.042202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:41553 (local address 127.17.44.195:44855)
0812 11:09:30.042421 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:30.042438 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:30.042531 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:30.042819 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:30.042822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:30.042839 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:09:30.043018 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:30.043025 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.043783 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.043786 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:30.044651 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:30.044661 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.045324 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.045328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:30.045442 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:30.045467 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 11:09:30.045555 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 11:09:30.045630 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121}
I20260812 11:09:30.045766 20172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:30.042284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:44855 (local address 127.17.44.194:41553)
0812 11:09:30.042502 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:30.042507 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:30.042521 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:09:30.042663 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:30.042671 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:30.042713 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:30.042796 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:09:30.043130 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.043657 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.044796 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.045193 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.045324 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:30.045379 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:30.045573 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 11:09:30.045649 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260812 11:09:30.046416 19844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92ab3c15b2ae4cfe847a88cc4fab3a2a" candidate_uuid: "77e67f84942f4e58a3990fed8e009e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "936988d8930a425ab67afb31c00c7538" is_pre_election: true
I20260812 11:09:30.046736 19844 raft_consensus.cc:2454] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 936988d8930a425ab67afb31c00c7538 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77e67f84942f4e58a3990fed8e009e85 in term 0.
I20260812 11:09:30.047272 19973 leader_election.cc:302] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77e67f84942f4e58a3990fed8e009e85, 936988d8930a425ab67afb31c00c7538; no voters:
I20260812 11:09:30.047698 20195 raft_consensus.cc:2781] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:09:30.047816 20195 raft_consensus.cc:482] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:09:30.047887 20195 raft_consensus.cc:3037] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:30.048170 20178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:30.042410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:57285 (local address 127.17.44.193:38151)
0812 11:09:30.042579 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:30.042584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:30.042887 (+ 303us) server_negotiation.cc:407] Connection header received
0812 11:09:30.043079 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:30.043084 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:30.043132 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:30.043231 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:09:30.043503 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.044008 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.046393 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.047131 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.047318 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:30.047432 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:30.047936 (+ 504us) server_negotiation.cc:299] Negotiation successful
0812 11:09:30.048068 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 11:09:30.048223 20201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:30.042239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:38151 (local address 127.17.44.195:57285)
0812 11:09:30.042762 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:30.042778 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:30.042959 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:30.043244 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:30.043247 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:30.043264 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:09:30.043392 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:30.043398 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.044117 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.044122 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:30.046139 (+ 2017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:30.046179 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.047464 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.047472 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:30.047731 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:30.047765 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 11:09:30.047875 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 11:09:30.047972 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":165,"threads_started":1}
I20260812 11:09:30.049310 19626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92ab3c15b2ae4cfe847a88cc4fab3a2a" candidate_uuid: "77e67f84942f4e58a3990fed8e009e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41759f922954da9b6d51cbfe3691a98" is_pre_election: true
I20260812 11:09:30.049861 19626 raft_consensus.cc:2454] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P f41759f922954da9b6d51cbfe3691a98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77e67f84942f4e58a3990fed8e009e85 in term 0.
I20260812 11:09:30.050362 20195 raft_consensus.cc:504] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:30.050849 20195 leader_election.cc:288] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85 [CANDIDATE]: Term 1 election: Requested vote from peers 936988d8930a425ab67afb31c00c7538 (127.17.44.194:41553), f41759f922954da9b6d51cbfe3691a98 (127.17.44.193:38151)
I20260812 11:09:30.051297 19844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92ab3c15b2ae4cfe847a88cc4fab3a2a" candidate_uuid: "77e67f84942f4e58a3990fed8e009e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "936988d8930a425ab67afb31c00c7538"
I20260812 11:09:30.051472 19844 raft_consensus.cc:3037] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 936988d8930a425ab67afb31c00c7538 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:30.051783 19626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92ab3c15b2ae4cfe847a88cc4fab3a2a" candidate_uuid: "77e67f84942f4e58a3990fed8e009e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41759f922954da9b6d51cbfe3691a98"
I20260812 11:09:30.051955 19626 raft_consensus.cc:3037] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P f41759f922954da9b6d51cbfe3691a98 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:30.053400 19844 raft_consensus.cc:2454] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 936988d8930a425ab67afb31c00c7538 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77e67f84942f4e58a3990fed8e009e85 in term 1.
I20260812 11:09:30.053783 19973 leader_election.cc:302] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77e67f84942f4e58a3990fed8e009e85, 936988d8930a425ab67afb31c00c7538; no voters:
I20260812 11:09:30.053983 20195 raft_consensus.cc:2781] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:30.054227 20195 raft_consensus.cc:686] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85 [term 1 LEADER]: Becoming Leader. State: Replica: 77e67f84942f4e58a3990fed8e009e85, State: Running, Role: LEADER
I20260812 11:09:30.054247 19626 raft_consensus.cc:2454] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P f41759f922954da9b6d51cbfe3691a98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77e67f84942f4e58a3990fed8e009e85 in term 1.
I20260812 11:09:30.054648 20195 consensus_queue.cc:237] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:30.058410 19371 catalog_manager.cc:5495] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 77e67f84942f4e58a3990fed8e009e85 (127.17.44.195). New cstate: current_term: 1 leader_uuid: "77e67f84942f4e58a3990fed8e009e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } health_report { overall_health: HEALTHY } } }
W20260812 11:09:30.064308 19926 tablet.cc:2369] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 936988d8930a425ab67afb31c00c7538: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:09:30.070827 19704 tablet.cc:2369] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P f41759f922954da9b6d51cbfe3691a98: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:09:30.076077 20149 tablet.cc:2369] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:09:30.094158 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:30.090082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55348 (local address 127.17.44.195:45919)
0812 11:09:30.090231 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:30.090234 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:30.090250 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:09:30.090473 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:30.090476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:30.090535 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:30.090625 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:09:30.091059 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.091585 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.092556 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.092791 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.092907 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:30.093250 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:09:30.093726 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:09:30.093862 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:30.093952 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 11:09:30.094010 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 11:09:30.101018 19626 raft_consensus.cc:1261] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P f41759f922954da9b6d51cbfe3691a98 [term 1 FOLLOWER]: Refusing update from remote peer 77e67f84942f4e58a3990fed8e009e85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:09:30.101987 20202 consensus_queue.cc:1035] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:09:30.102564 19844 raft_consensus.cc:1261] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 936988d8930a425ab67afb31c00c7538 [term 1 FOLLOWER]: Refusing update from remote peer 77e67f84942f4e58a3990fed8e009e85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:09:30.103364 20202 consensus_queue.cc:1035] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:09:30.110123 20211 mvcc.cc:205] Tried to move back new op lower bound from 7317639045523611648 to 7317639045347045376. Current Snapshot: MvccSnapshot[applied={T|T < 7317639045523611648}]
I20260812 11:09:30.110500 20210 mvcc.cc:205] Tried to move back new op lower bound from 7317639045523611648 to 7317639045347045376. Current Snapshot: MvccSnapshot[applied={T|T < 7317639045523611648}]
I20260812 11:09:30.113786 20213 mvcc.cc:205] Tried to move back new op lower bound from 7317639045523611648 to 7317639045347045376. Current Snapshot: MvccSnapshot[applied={T|T < 7317639045523611648}]
I20260812 11:09:30.125715 19379 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:09:30.125955 19379 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:09:30.129166 19379 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda: Sending DeleteTablet for 3 replicas of tablet 92ab3c15b2ae4cfe847a88cc4fab3a2a
I20260812 11:09:30.130302 19819 tablet_service.cc:1514] Processing DeleteTablet for tablet 92ab3c15b2ae4cfe847a88cc4fab3a2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:09:30 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:50456
I20260812 11:09:30.130455 19596 tablet_service.cc:1514] Processing DeleteTablet for tablet 92ab3c15b2ae4cfe847a88cc4fab3a2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:09:30 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:51518
I20260812 11:09:30.131028 20226 tablet_replica.cc:330] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P f41759f922954da9b6d51cbfe3691a98: stopping tablet replica
I20260812 11:09:30.131004 20036 tablet_service.cc:1514] Processing DeleteTablet for tablet 92ab3c15b2ae4cfe847a88cc4fab3a2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:09:30 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:55338
I20260812 11:09:30.131278 19379 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60424:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 11:09:30.131397 20226 raft_consensus.cc:2229] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P f41759f922954da9b6d51cbfe3691a98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:09:30.131628 20227 tablet_replica.cc:330] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85: stopping tablet replica
W20260812 11:09:30.131677 19379 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:09:30.131742 20226 raft_consensus.cc:2258] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P f41759f922954da9b6d51cbfe3691a98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:09:30.131897 20227 raft_consensus.cc:2229] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:09:30.132288 20227 raft_consensus.cc:2258] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:09:30.132778 20225 tablet_replica.cc:330] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 936988d8930a425ab67afb31c00c7538: stopping tablet replica
I20260812 11:09:30.133081 20225 raft_consensus.cc:2229] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 936988d8930a425ab67afb31c00c7538 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:09:30.133373 20225 raft_consensus.cc:2258] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 936988d8930a425ab67afb31c00c7538 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:09:30.134195 20227 ts_tablet_manager.cc:1905] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:09:30.134634 20225 ts_tablet_manager.cc:1905] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 936988d8930a425ab67afb31c00c7538: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:09:30.138257 20227 ts_tablet_manager.cc:1918] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:09:30.138356 19596 tablet_service.cc:1467] Processing CreateTablet for tablet 05487855a8e0460b8ded8d4d209b39d6 (DEFAULT_TABLE table=test-table [id=d7e0c7963e604645bb0b9ad181023775]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:30.138568 20227 log.cc:1196] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/92ab3c15b2ae4cfe847a88cc4fab3a2a
I20260812 11:09:30.138816 19596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05487855a8e0460b8ded8d4d209b39d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:30.139321 20227 ts_tablet_manager.cc:1939] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 77e67f84942f4e58a3990fed8e009e85: Deleting consensus metadata
I20260812 11:09:30.141054 19358 catalog_manager.cc:4841] TS 77e67f84942f4e58a3990fed8e009e85 (127.17.44.195:45919): tablet 92ab3c15b2ae4cfe847a88cc4fab3a2a (table test-table [id=27f12243487549c7a71877654401ff1d]) successfully deleted
I20260812 11:09:30.141835 20183 tablet_bootstrap.cc:492] T 05487855a8e0460b8ded8d4d209b39d6 P f41759f922954da9b6d51cbfe3691a98: Bootstrap starting.
I20260812 11:09:30.142113 20225 ts_tablet_manager.cc:1918] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 936988d8930a425ab67afb31c00c7538: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:09:30.142222 20225 log.cc:1196] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 936988d8930a425ab67afb31c00c7538: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/92ab3c15b2ae4cfe847a88cc4fab3a2a
I20260812 11:09:30.142083 20036 tablet_service.cc:1467] Processing CreateTablet for tablet 05487855a8e0460b8ded8d4d209b39d6 (DEFAULT_TABLE table=test-table [id=d7e0c7963e604645bb0b9ad181023775]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:30.142429 20036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05487855a8e0460b8ded8d4d209b39d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:30.142583 20225 ts_tablet_manager.cc:1939] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P 936988d8930a425ab67afb31c00c7538: Deleting consensus metadata
I20260812 11:09:30.143230 20183 tablet_bootstrap.cc:654] T 05487855a8e0460b8ded8d4d209b39d6 P f41759f922954da9b6d51cbfe3691a98: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:30.144963 19819 tablet_service.cc:1467] Processing CreateTablet for tablet 05487855a8e0460b8ded8d4d209b39d6 (DEFAULT_TABLE table=test-table [id=d7e0c7963e604645bb0b9ad181023775]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:30.145213 19819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05487855a8e0460b8ded8d4d209b39d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:30.145568 20183 tablet_bootstrap.cc:492] T 05487855a8e0460b8ded8d4d209b39d6 P f41759f922954da9b6d51cbfe3691a98: No bootstrap required, opened a new log
I20260812 11:09:30.145665 20183 ts_tablet_manager.cc:1397] T 05487855a8e0460b8ded8d4d209b39d6 P f41759f922954da9b6d51cbfe3691a98: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 11:09:30.146078 20183 raft_consensus.cc:348] T 05487855a8e0460b8ded8d4d209b39d6 P f41759f922954da9b6d51cbfe3691a98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:30.146204 20183 raft_consensus.cc:374] T 05487855a8e0460b8ded8d4d209b39d6 P f41759f922954da9b6d51cbfe3691a98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:30.146255 20183 raft_consensus.cc:729] T 05487855a8e0460b8ded8d4d209b39d6 P f41759f922954da9b6d51cbfe3691a98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41759f922954da9b6d51cbfe3691a98, State: Initialized, Role: FOLLOWER
I20260812 11:09:30.146376 20183 consensus_queue.cc:260] T 05487855a8e0460b8ded8d4d209b39d6 P f41759f922954da9b6d51cbfe3691a98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:30.146648 20183 ts_tablet_manager.cc:1428] T 05487855a8e0460b8ded8d4d209b39d6 P f41759f922954da9b6d51cbfe3691a98: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 11:09:30.147451 20186 tablet_bootstrap.cc:492] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538: Bootstrap starting.
I20260812 11:09:30.147534 20184 tablet_bootstrap.cc:492] T 05487855a8e0460b8ded8d4d209b39d6 P 77e67f84942f4e58a3990fed8e009e85: Bootstrap starting.
I20260812 11:09:30.148485 20184 tablet_bootstrap.cc:654] T 05487855a8e0460b8ded8d4d209b39d6 P 77e67f84942f4e58a3990fed8e009e85: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:30.148586 20186 tablet_bootstrap.cc:654] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:30.149394 19362 catalog_manager.cc:4841] TS 936988d8930a425ab67afb31c00c7538 (127.17.44.194:41553): tablet 92ab3c15b2ae4cfe847a88cc4fab3a2a (table test-table [id=27f12243487549c7a71877654401ff1d]) successfully deleted
I20260812 11:09:30.150378 20186 tablet_bootstrap.cc:492] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538: No bootstrap required, opened a new log
I20260812 11:09:30.150466 20186 ts_tablet_manager.cc:1397] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 11:09:30.151085 20226 ts_tablet_manager.cc:1905] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P f41759f922954da9b6d51cbfe3691a98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:09:30.151407 20186 raft_consensus.cc:348] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:30.151537 20186 raft_consensus.cc:374] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:30.151578 20186 raft_consensus.cc:729] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 936988d8930a425ab67afb31c00c7538, State: Initialized, Role: FOLLOWER
I20260812 11:09:30.151722 20186 consensus_queue.cc:260] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:30.152096 20186 ts_tablet_manager.cc:1428] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260812 11:09:30.153508 20184 tablet_bootstrap.cc:492] T 05487855a8e0460b8ded8d4d209b39d6 P 77e67f84942f4e58a3990fed8e009e85: No bootstrap required, opened a new log
I20260812 11:09:30.153596 20184 ts_tablet_manager.cc:1397] T 05487855a8e0460b8ded8d4d209b39d6 P 77e67f84942f4e58a3990fed8e009e85: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260812 11:09:30.154040 20184 raft_consensus.cc:348] T 05487855a8e0460b8ded8d4d209b39d6 P 77e67f84942f4e58a3990fed8e009e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:30.154121 20184 raft_consensus.cc:374] T 05487855a8e0460b8ded8d4d209b39d6 P 77e67f84942f4e58a3990fed8e009e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:30.154142 20184 raft_consensus.cc:729] T 05487855a8e0460b8ded8d4d209b39d6 P 77e67f84942f4e58a3990fed8e009e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e67f84942f4e58a3990fed8e009e85, State: Initialized, Role: FOLLOWER
I20260812 11:09:30.154276 20184 consensus_queue.cc:260] T 05487855a8e0460b8ded8d4d209b39d6 P 77e67f84942f4e58a3990fed8e009e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:30.154541 20184 ts_tablet_manager.cc:1428] T 05487855a8e0460b8ded8d4d209b39d6 P 77e67f84942f4e58a3990fed8e009e85: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 11:09:30.155615 20226 ts_tablet_manager.cc:1918] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P f41759f922954da9b6d51cbfe3691a98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:09:30.155720 20226 log.cc:1196] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P f41759f922954da9b6d51cbfe3691a98: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-0/wal/wals/92ab3c15b2ae4cfe847a88cc4fab3a2a
I20260812 11:09:30.156121 20226 ts_tablet_manager.cc:1939] T 92ab3c15b2ae4cfe847a88cc4fab3a2a P f41759f922954da9b6d51cbfe3691a98: Deleting consensus metadata
I20260812 11:09:30.157394 19358 catalog_manager.cc:4841] TS f41759f922954da9b6d51cbfe3691a98 (127.17.44.193:38151): tablet 92ab3c15b2ae4cfe847a88cc4fab3a2a (table test-table [id=27f12243487549c7a71877654401ff1d]) successfully deleted
I20260812 11:09:30.559201 20196 raft_consensus.cc:482] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:09:30.559482 20196 raft_consensus.cc:504] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:30.560702 20196 leader_election.cc:288] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f41759f922954da9b6d51cbfe3691a98 (127.17.44.193:38151), 77e67f84942f4e58a3990fed8e009e85 (127.17.44.195:45919)
I20260812 11:09:30.566107 20237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:30.561409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:45919 (local address 127.17.44.194:47759)
0812 11:09:30.561774 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:30.561825 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:30.562022 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:30.562465 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:30.562467 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:30.562487 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:09:30.562601 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:30.562607 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.563754 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.563759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:30.564602 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:30.564612 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.565586 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.565590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:30.565702 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:30.565793 (+ 91us) client_negotiation.cc:769] Sending connection context
0812 11:09:30.565883 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 11:09:30.565957 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":52,"thread_start_us":82,"threads_started":1}
I20260812 11:09:30.566140 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:30.561629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:47759 (local address 127.17.44.195:45919)
0812 11:09:30.561790 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:30.561794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:30.561976 (+ 182us) server_negotiation.cc:407] Connection header received
0812 11:09:30.562244 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:30.562247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:30.562342 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:30.562440 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:09:30.562740 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.563611 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.564740 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.565416 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.565538 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:30.565601 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:30.565918 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 11:09:30.566011 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 11:09:30.566428 20233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:30.561004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:38151 (local address 127.17.44.194:57791)
0812 11:09:30.561267 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:30.561284 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:30.561415 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:30.562055 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:30.562058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:30.562083 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:09:30.562207 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:30.562214 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.563609 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.563634 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:30.564747 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:30.564759 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.565704 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.565708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:30.565864 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:30.565895 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 11:09:30.565982 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 11:09:30.566047 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":3,"thread_start_us":112,"threads_started":1}
I20260812 11:09:30.566664 20067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05487855a8e0460b8ded8d4d209b39d6" candidate_uuid: "936988d8930a425ab67afb31c00c7538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e67f84942f4e58a3990fed8e009e85" is_pre_election: true
I20260812 11:09:30.566857 20235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:30.561190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:57791 (local address 127.17.44.193:38151)
0812 11:09:30.561636 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:30.561642 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:30.561665 (+ 23us) server_negotiation.cc:407] Connection header received
0812 11:09:30.561759 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:30.561763 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:30.561841 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:30.561987 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:09:30.562363 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.563456 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.564886 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.565568 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.565693 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:30.565753 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:30.566157 (+ 404us) server_negotiation.cc:299] Negotiation successful
0812 11:09:30.566720 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":158,"threads_started":1}
I20260812 11:09:30.566905 20067 raft_consensus.cc:2454] T 05487855a8e0460b8ded8d4d209b39d6 P 77e67f84942f4e58a3990fed8e009e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 936988d8930a425ab67afb31c00c7538 in term 0.
I20260812 11:09:30.567261 19626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05487855a8e0460b8ded8d4d209b39d6" candidate_uuid: "936988d8930a425ab67afb31c00c7538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41759f922954da9b6d51cbfe3691a98" is_pre_election: true
I20260812 11:09:30.567337 19741 leader_election.cc:302] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77e67f84942f4e58a3990fed8e009e85, 936988d8930a425ab67afb31c00c7538; no voters:
I20260812 11:09:30.567409 19626 raft_consensus.cc:2454] T 05487855a8e0460b8ded8d4d209b39d6 P f41759f922954da9b6d51cbfe3691a98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 936988d8930a425ab67afb31c00c7538 in term 0.
I20260812 11:09:30.567643 20196 raft_consensus.cc:2781] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:09:30.567708 20196 raft_consensus.cc:482] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:09:30.567770 20196 raft_consensus.cc:3037] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:30.568948 20196 raft_consensus.cc:504] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:30.569324 20196 leader_election.cc:288] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538 [CANDIDATE]: Term 1 election: Requested vote from peers f41759f922954da9b6d51cbfe3691a98 (127.17.44.193:38151), 77e67f84942f4e58a3990fed8e009e85 (127.17.44.195:45919)
I20260812 11:09:30.569626 19626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05487855a8e0460b8ded8d4d209b39d6" candidate_uuid: "936988d8930a425ab67afb31c00c7538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41759f922954da9b6d51cbfe3691a98"
I20260812 11:09:30.569698 20067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05487855a8e0460b8ded8d4d209b39d6" candidate_uuid: "936988d8930a425ab67afb31c00c7538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e67f84942f4e58a3990fed8e009e85"
I20260812 11:09:30.569769 19626 raft_consensus.cc:3037] T 05487855a8e0460b8ded8d4d209b39d6 P f41759f922954da9b6d51cbfe3691a98 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:30.569836 20067 raft_consensus.cc:3037] T 05487855a8e0460b8ded8d4d209b39d6 P 77e67f84942f4e58a3990fed8e009e85 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:30.570657 19626 raft_consensus.cc:2454] T 05487855a8e0460b8ded8d4d209b39d6 P f41759f922954da9b6d51cbfe3691a98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 936988d8930a425ab67afb31c00c7538 in term 1.
I20260812 11:09:30.570961 19741 leader_election.cc:302] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 936988d8930a425ab67afb31c00c7538, f41759f922954da9b6d51cbfe3691a98; no voters:
I20260812 11:09:30.571142 20196 raft_consensus.cc:2781] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:30.571182 20067 raft_consensus.cc:2454] T 05487855a8e0460b8ded8d4d209b39d6 P 77e67f84942f4e58a3990fed8e009e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 936988d8930a425ab67afb31c00c7538 in term 1.
I20260812 11:09:30.571358 20196 raft_consensus.cc:686] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538 [term 1 LEADER]: Becoming Leader. State: Replica: 936988d8930a425ab67afb31c00c7538, State: Running, Role: LEADER
I20260812 11:09:30.571627 20196 consensus_queue.cc:237] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:30.574244 19371 catalog_manager.cc:5495] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538 reported cstate change: term changed from 0 to 1, leader changed from <none> to 936988d8930a425ab67afb31c00c7538 (127.17.44.194). New cstate: current_term: 1 leader_uuid: "936988d8930a425ab67afb31c00c7538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } health_report { overall_health: UNKNOWN } } }
Aug 12 11:09:30 dist-test-slave-drl0 krb5kdc[19316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532968, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:30.631270 20248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:30.621041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42883 (local address 127.0.0.1:60428)
0812 11:09:30.621384 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:30.621398 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:30.621505 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:30.621805 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:30.621808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:30.622211 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:30.622307 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:30.622313 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.623191 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.623193 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:30.623742 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:30.623750 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.624250 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.624253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:30.624331 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:30.625318 (+ 987us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:30.625338 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:30.627326 (+ 1988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:30.629327 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:30.629342 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:30.629348 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:30.629602 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:30.629951 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:30.629954 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:30.629958 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:30.630107 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:30.630551 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:30.630558 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:30.630801 (+ 243us) client_negotiation.cc:769] Sending connection context
0812 11:09:30.630910 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 11:09:30.631101 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":92,"threads_started":1}
I20260812 11:09:30.631446 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:30.621149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60428 (local address 127.17.44.254:42883)
0812 11:09:30.621441 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:30.621449 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:30.621465 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:09:30.621624 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:30.621627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:30.621674 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:30.621762 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:30.622442 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.623066 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.623867 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.624121 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.624216 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:30.627481 (+ 3265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:30.627484 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:30.627486 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:30.627509 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:30.629202 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:30.629704 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:30.629707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:30.629708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:30.629801 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:30.630228 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:30.630231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:30.630232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:30.630415 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:30.630514 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:30.631070 (+ 556us) server_negotiation.cc:299] Negotiation successful
0812 11:09:30.631177 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":103,"threads_started":1}
I20260812 11:09:30.635846 19371 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:60428:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 11:09:30.636718 19371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:09:30.642887 19596 tablet_service.cc:1467] Processing CreateTablet for tablet f9d10237de104632bc600219c2d16e52 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=20a40909439142d6a217450d0c745a88]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 11:09:30.643049 19819 tablet_service.cc:1467] Processing CreateTablet for tablet f9d10237de104632bc600219c2d16e52 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=20a40909439142d6a217450d0c745a88]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 11:09:30.643049 20036 tablet_service.cc:1467] Processing CreateTablet for tablet f9d10237de104632bc600219c2d16e52 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=20a40909439142d6a217450d0c745a88]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 11:09:30.643432 19596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9d10237de104632bc600219c2d16e52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:30.643431 20036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9d10237de104632bc600219c2d16e52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:30.643431 19819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9d10237de104632bc600219c2d16e52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:30.646430 20186 tablet_bootstrap.cc:492] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538: Bootstrap starting.
I20260812 11:09:30.647310 20186 tablet_bootstrap.cc:654] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:30.647944 20184 tablet_bootstrap.cc:492] T f9d10237de104632bc600219c2d16e52 P 77e67f84942f4e58a3990fed8e009e85: Bootstrap starting.
I20260812 11:09:30.648288 20251 tablet_bootstrap.cc:492] T f9d10237de104632bc600219c2d16e52 P f41759f922954da9b6d51cbfe3691a98: Bootstrap starting.
I20260812 11:09:30.648941 20184 tablet_bootstrap.cc:654] T f9d10237de104632bc600219c2d16e52 P 77e67f84942f4e58a3990fed8e009e85: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:30.649375 20251 tablet_bootstrap.cc:654] T f9d10237de104632bc600219c2d16e52 P f41759f922954da9b6d51cbfe3691a98: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:30.650170 20186 tablet_bootstrap.cc:492] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538: No bootstrap required, opened a new log
I20260812 11:09:30.650183 20184 tablet_bootstrap.cc:492] T f9d10237de104632bc600219c2d16e52 P 77e67f84942f4e58a3990fed8e009e85: No bootstrap required, opened a new log
I20260812 11:09:30.650279 20186 ts_tablet_manager.cc:1397] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 11:09:30.650298 20184 ts_tablet_manager.cc:1397] T f9d10237de104632bc600219c2d16e52 P 77e67f84942f4e58a3990fed8e009e85: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 11:09:30.650741 20186 raft_consensus.cc:348] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:30.650801 20184 raft_consensus.cc:348] T f9d10237de104632bc600219c2d16e52 P 77e67f84942f4e58a3990fed8e009e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:30.650894 20186 raft_consensus.cc:374] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:30.650928 20184 raft_consensus.cc:374] T f9d10237de104632bc600219c2d16e52 P 77e67f84942f4e58a3990fed8e009e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:30.650959 20186 raft_consensus.cc:729] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 936988d8930a425ab67afb31c00c7538, State: Initialized, Role: FOLLOWER
I20260812 11:09:30.650995 20184 raft_consensus.cc:729] T f9d10237de104632bc600219c2d16e52 P 77e67f84942f4e58a3990fed8e009e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e67f84942f4e58a3990fed8e009e85, State: Initialized, Role: FOLLOWER
I20260812 11:09:30.651046 20251 tablet_bootstrap.cc:492] T f9d10237de104632bc600219c2d16e52 P f41759f922954da9b6d51cbfe3691a98: No bootstrap required, opened a new log
I20260812 11:09:30.651099 20186 consensus_queue.cc:260] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:30.651194 20251 ts_tablet_manager.cc:1397] T f9d10237de104632bc600219c2d16e52 P f41759f922954da9b6d51cbfe3691a98: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260812 11:09:30.651140 20184 consensus_queue.cc:260] T f9d10237de104632bc600219c2d16e52 P 77e67f84942f4e58a3990fed8e009e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:30.651396 20186 ts_tablet_manager.cc:1428] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 11:09:30.651499 20184 ts_tablet_manager.cc:1428] T f9d10237de104632bc600219c2d16e52 P 77e67f84942f4e58a3990fed8e009e85: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 11:09:30.651476 20202 tablet_replica.cc:439] T f9d10237de104632bc600219c2d16e52 P 77e67f84942f4e58a3990fed8e009e85: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } } }
I20260812 11:09:30.651616 20202 tablet_replica.cc:442] T f9d10237de104632bc600219c2d16e52 P 77e67f84942f4e58a3990fed8e009e85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:09:30.651782 20196 tablet_replica.cc:439] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } } }
I20260812 11:09:30.651906 20196 tablet_replica.cc:442] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:09:30.652478 20251 raft_consensus.cc:348] T f9d10237de104632bc600219c2d16e52 P f41759f922954da9b6d51cbfe3691a98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:30.652621 20251 raft_consensus.cc:374] T f9d10237de104632bc600219c2d16e52 P f41759f922954da9b6d51cbfe3691a98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:30.653460 20251 raft_consensus.cc:729] T f9d10237de104632bc600219c2d16e52 P f41759f922954da9b6d51cbfe3691a98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41759f922954da9b6d51cbfe3691a98, State: Initialized, Role: FOLLOWER
I20260812 11:09:30.653652 20251 consensus_queue.cc:260] T f9d10237de104632bc600219c2d16e52 P f41759f922954da9b6d51cbfe3691a98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:30.654083 20255 tablet_replica.cc:439] T f9d10237de104632bc600219c2d16e52 P f41759f922954da9b6d51cbfe3691a98: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } } }
I20260812 11:09:30.654101 20251 ts_tablet_manager.cc:1428] T f9d10237de104632bc600219c2d16e52 P f41759f922954da9b6d51cbfe3691a98: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260812 11:09:30.654228 20255 tablet_replica.cc:442] T f9d10237de104632bc600219c2d16e52 P f41759f922954da9b6d51cbfe3691a98: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:09:30.655349 20196 raft_consensus.cc:482] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:09:30.655457 20196 raft_consensus.cc:504] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:30.655869 20196 leader_election.cc:288] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f41759f922954da9b6d51cbfe3691a98 (127.17.44.193:38151), 77e67f84942f4e58a3990fed8e009e85 (127.17.44.195:45919)
I20260812 11:09:30.656306 19626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9d10237de104632bc600219c2d16e52" candidate_uuid: "936988d8930a425ab67afb31c00c7538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41759f922954da9b6d51cbfe3691a98" is_pre_election: true
I20260812 11:09:30.656296 20067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9d10237de104632bc600219c2d16e52" candidate_uuid: "936988d8930a425ab67afb31c00c7538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e67f84942f4e58a3990fed8e009e85" is_pre_election: true
I20260812 11:09:30.656430 20067 raft_consensus.cc:2454] T f9d10237de104632bc600219c2d16e52 P 77e67f84942f4e58a3990fed8e009e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 936988d8930a425ab67afb31c00c7538 in term 0.
I20260812 11:09:30.656443 19626 raft_consensus.cc:2454] T f9d10237de104632bc600219c2d16e52 P f41759f922954da9b6d51cbfe3691a98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 936988d8930a425ab67afb31c00c7538 in term 0.
I20260812 11:09:30.656701 19741 leader_election.cc:302] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77e67f84942f4e58a3990fed8e009e85, 936988d8930a425ab67afb31c00c7538; no voters:
I20260812 11:09:30.656874 20196 raft_consensus.cc:2781] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:09:30.656931 20196 raft_consensus.cc:482] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:09:30.656970 20196 raft_consensus.cc:3037] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:30.657897 20196 raft_consensus.cc:504] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:30.658218 20196 leader_election.cc:288] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538 [CANDIDATE]: Term 1 election: Requested vote from peers f41759f922954da9b6d51cbfe3691a98 (127.17.44.193:38151), 77e67f84942f4e58a3990fed8e009e85 (127.17.44.195:45919)
I20260812 11:09:30.658594 20067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9d10237de104632bc600219c2d16e52" candidate_uuid: "936988d8930a425ab67afb31c00c7538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e67f84942f4e58a3990fed8e009e85"
I20260812 11:09:30.658730 20067 raft_consensus.cc:3037] T f9d10237de104632bc600219c2d16e52 P 77e67f84942f4e58a3990fed8e009e85 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:30.658732 19626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9d10237de104632bc600219c2d16e52" candidate_uuid: "936988d8930a425ab67afb31c00c7538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41759f922954da9b6d51cbfe3691a98"
I20260812 11:09:30.658955 19626 raft_consensus.cc:3037] T f9d10237de104632bc600219c2d16e52 P f41759f922954da9b6d51cbfe3691a98 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:30.659989 20067 raft_consensus.cc:2454] T f9d10237de104632bc600219c2d16e52 P 77e67f84942f4e58a3990fed8e009e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 936988d8930a425ab67afb31c00c7538 in term 1.
I20260812 11:09:30.659989 19626 raft_consensus.cc:2454] T f9d10237de104632bc600219c2d16e52 P f41759f922954da9b6d51cbfe3691a98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 936988d8930a425ab67afb31c00c7538 in term 1.
I20260812 11:09:30.660275 19741 leader_election.cc:302] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 936988d8930a425ab67afb31c00c7538, f41759f922954da9b6d51cbfe3691a98; no voters:
I20260812 11:09:30.660477 20196 raft_consensus.cc:2781] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:30.660593 20196 raft_consensus.cc:686] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538 [term 1 LEADER]: Becoming Leader. State: Replica: 936988d8930a425ab67afb31c00c7538, State: Running, Role: LEADER
I20260812 11:09:30.660748 20196 consensus_queue.cc:237] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } }
I20260812 11:09:30.661439 20238 tablet_replica.cc:439] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538: TxnStatusTablet state changed. Reason: New leader 936988d8930a425ab67afb31c00c7538. Latest consensus state: current_term: 1 leader_uuid: "936988d8930a425ab67afb31c00c7538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } } }
I20260812 11:09:30.661559 20238 tablet_replica.cc:442] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538: This TxnStatusTablet replica's current role is: LEADER
I20260812 11:09:30.661852 20257 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 11:09:30.662046 20257 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 11:09:30.662523 19379 catalog_manager.cc:5495] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538 reported cstate change: term changed from 0 to 1, leader changed from <none> to 936988d8930a425ab67afb31c00c7538 (127.17.44.194). New cstate: current_term: 1 leader_uuid: "936988d8930a425ab67afb31c00c7538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } health_report { overall_health: UNKNOWN } } }
I20260812 11:09:30.680769 20248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:30.676288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:41553 (local address 127.0.0.1:50464)
0812 11:09:30.676442 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:30.676459 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:30.676594 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:30.677012 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:30.677015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:30.677023 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 11:09:30.677130 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:30.677136 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.678022 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.678027 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:30.678815 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:30.678827 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.679522 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.679525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:30.679636 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:30.679683 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 11:09:30.680439 (+ 756us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 11:09:30.680457 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 11:09:30.680536 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 11:09:30.680627 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 11:09:30.680864 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:30.676397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50464 (local address 127.17.44.194:41553)
0812 11:09:30.676703 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:30.676713 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:30.676734 (+ 21us) server_negotiation.cc:407] Connection header received
0812 11:09:30.676789 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:30.676791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:30.676838 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:30.676999 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:09:30.677248 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.677901 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.678991 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.679384 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.679511 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:30.679850 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:09:30.680334 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:09:30.680463 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:30.680547 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 11:09:30.680602 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":107,"threads_started":1}
I20260812 11:09:30.684379 19626 raft_consensus.cc:1261] T f9d10237de104632bc600219c2d16e52 P f41759f922954da9b6d51cbfe3691a98 [term 1 FOLLOWER]: Refusing update from remote peer 936988d8930a425ab67afb31c00c7538: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:09:30.684634 20067 raft_consensus.cc:1261] T f9d10237de104632bc600219c2d16e52 P 77e67f84942f4e58a3990fed8e009e85 [term 1 FOLLOWER]: Refusing update from remote peer 936988d8930a425ab67afb31c00c7538: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:09:30.685112 20196 consensus_queue.cc:1035] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:09:30.685535 20238 consensus_queue.cc:1035] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:09:30.686815 20255 tablet_replica.cc:439] T f9d10237de104632bc600219c2d16e52 P f41759f922954da9b6d51cbfe3691a98: TxnStatusTablet state changed. Reason: New leader 936988d8930a425ab67afb31c00c7538. Latest consensus state: current_term: 1 leader_uuid: "936988d8930a425ab67afb31c00c7538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } } }
I20260812 11:09:30.686981 20255 tablet_replica.cc:442] T f9d10237de104632bc600219c2d16e52 P f41759f922954da9b6d51cbfe3691a98: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:09:30.688187 20202 tablet_replica.cc:439] T f9d10237de104632bc600219c2d16e52 P 77e67f84942f4e58a3990fed8e009e85: TxnStatusTablet state changed. Reason: New leader 936988d8930a425ab67afb31c00c7538. Latest consensus state: current_term: 1 leader_uuid: "936988d8930a425ab67afb31c00c7538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } } }
I20260812 11:09:30.688424 20202 tablet_replica.cc:442] T f9d10237de104632bc600219c2d16e52 P 77e67f84942f4e58a3990fed8e009e85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:09:30.691624 20238 tablet_replica.cc:439] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "936988d8930a425ab67afb31c00c7538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } } }
I20260812 11:09:30.691793 20238 tablet_replica.cc:442] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538: This TxnStatusTablet replica's current role is: LEADER
I20260812 11:09:30.691741 20196 tablet_replica.cc:439] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "936988d8930a425ab67afb31c00c7538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } } }
I20260812 11:09:30.691896 20196 tablet_replica.cc:442] T f9d10237de104632bc600219c2d16e52 P 936988d8930a425ab67afb31c00c7538: This TxnStatusTablet replica's current role is: LEADER
I20260812 11:09:30.693239 20255 tablet_replica.cc:439] T f9d10237de104632bc600219c2d16e52 P f41759f922954da9b6d51cbfe3691a98: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "936988d8930a425ab67afb31c00c7538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } } }
I20260812 11:09:30.693396 20255 tablet_replica.cc:442] T f9d10237de104632bc600219c2d16e52 P f41759f922954da9b6d51cbfe3691a98: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:09:30.694622 20202 tablet_replica.cc:439] T f9d10237de104632bc600219c2d16e52 P 77e67f84942f4e58a3990fed8e009e85: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "936988d8930a425ab67afb31c00c7538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 } } peers { permanent_uuid: "936988d8930a425ab67afb31c00c7538" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41553 } } peers { permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 } } }
I20260812 11:09:30.694763 20202 tablet_replica.cc:442] T f9d10237de104632bc600219c2d16e52 P 77e67f84942f4e58a3990fed8e009e85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:09:30.706571 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:30.703196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50480 (local address 127.17.44.194:41553)
0812 11:09:30.703339 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:30.703343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:30.703567 (+ 224us) server_negotiation.cc:407] Connection header received
0812 11:09:30.703733 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:30.703737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:30.703790 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:30.703876 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:09:30.704156 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.704659 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.705519 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.705735 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.705845 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:30.706109 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:09:30.706186 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:09:30.706267 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:30.706371 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 11:09:30.706427 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 11:09:30.718801 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:30.714296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:34919 (local address 127.17.44.194:41553)
0812 11:09:30.714423 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:30.714427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:30.714666 (+ 239us) server_negotiation.cc:407] Connection header received
0812 11:09:30.714764 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:30.714767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:30.714805 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:30.714861 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:09:30.715290 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.716062 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.717083 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.717487 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.717609 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:30.717969 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:09:30.718060 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:09:30.718187 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:30.718590 (+ 403us) server_negotiation.cc:299] Negotiation successful
0812 11:09:30.718646 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 11:09:30.718852 20280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:30.714223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:41553 (local address 127.17.44.194:34919)
0812 11:09:30.714546 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:30.714563 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:30.714668 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:30.714892 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:30.714895 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:30.714903 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 11:09:30.715067 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:30.715073 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.716263 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.716266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:30.716929 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:30.716938 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.717640 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.717643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:30.717751 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:30.717802 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 11:09:30.718454 (+ 652us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 11:09:30.718460 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 11:09:30.718532 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 11:09:30.718593 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":144,"threads_started":1}
I20260812 11:09:30.726728 20067 raft_consensus.cc:1261] T 05487855a8e0460b8ded8d4d209b39d6 P 77e67f84942f4e58a3990fed8e009e85 [term 1 FOLLOWER]: Refusing update from remote peer 936988d8930a425ab67afb31c00c7538: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:09:30.726732 19626 raft_consensus.cc:1261] T 05487855a8e0460b8ded8d4d209b39d6 P f41759f922954da9b6d51cbfe3691a98 [term 1 FOLLOWER]: Refusing update from remote peer 936988d8930a425ab67afb31c00c7538: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:09:30.727669 20268 consensus_queue.cc:1035] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41759f922954da9b6d51cbfe3691a98" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 38151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:09:30.727837 20196 consensus_queue.cc:1035] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77e67f84942f4e58a3990fed8e009e85" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:09:30.772260 19379 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:09:30.772450 19379 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:09:30.774582 19379 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c50ee1e817b74c8ab9ad1a0f5594bcda: Sending DeleteTablet for 3 replicas of tablet 05487855a8e0460b8ded8d4d209b39d6
I20260812 11:09:30.775517 20036 tablet_service.cc:1514] Processing DeleteTablet for tablet 05487855a8e0460b8ded8d4d209b39d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:09:30 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:55338
I20260812 11:09:30.775561 19819 tablet_service.cc:1514] Processing DeleteTablet for tablet 05487855a8e0460b8ded8d4d209b39d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:09:30 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:50456
I20260812 11:09:30.775576 19596 tablet_service.cc:1514] Processing DeleteTablet for tablet 05487855a8e0460b8ded8d4d209b39d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:09:30 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:51518
I20260812 11:09:30.776328 20290 tablet_replica.cc:330] T 05487855a8e0460b8ded8d4d209b39d6 P f41759f922954da9b6d51cbfe3691a98: stopping tablet replica
I20260812 11:09:30.776513 20290 raft_consensus.cc:2229] T 05487855a8e0460b8ded8d4d209b39d6 P f41759f922954da9b6d51cbfe3691a98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:09:30.776629 20290 raft_consensus.cc:2258] T 05487855a8e0460b8ded8d4d209b39d6 P f41759f922954da9b6d51cbfe3691a98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:09:30.776959 20288 tablet_replica.cc:330] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538: stopping tablet replica
I20260812 11:09:30.777172 20288 raft_consensus.cc:2229] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:09:30.777357 20290 ts_tablet_manager.cc:1905] T 05487855a8e0460b8ded8d4d209b39d6 P f41759f922954da9b6d51cbfe3691a98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:09:30.777446 20287 tablet_replica.cc:330] T 05487855a8e0460b8ded8d4d209b39d6 P 77e67f84942f4e58a3990fed8e009e85: stopping tablet replica
I20260812 11:09:30.777601 20288 raft_consensus.cc:2258] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:09:30.778096 20287 raft_consensus.cc:2229] T 05487855a8e0460b8ded8d4d209b39d6 P 77e67f84942f4e58a3990fed8e009e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:09:30.778334 20288 ts_tablet_manager.cc:1905] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:09:30.778842 20287 raft_consensus.cc:2258] T 05487855a8e0460b8ded8d4d209b39d6 P 77e67f84942f4e58a3990fed8e009e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:09:30.780922 20287 ts_tablet_manager.cc:1905] T 05487855a8e0460b8ded8d4d209b39d6 P 77e67f84942f4e58a3990fed8e009e85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:09:30.781804 20290 ts_tablet_manager.cc:1918] T 05487855a8e0460b8ded8d4d209b39d6 P f41759f922954da9b6d51cbfe3691a98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:09:30.781971 20288 ts_tablet_manager.cc:1918] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:09:30.782050 20288 log.cc:1196] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/05487855a8e0460b8ded8d4d209b39d6
I20260812 11:09:30.781957 20290 log.cc:1196] T 05487855a8e0460b8ded8d4d209b39d6 P f41759f922954da9b6d51cbfe3691a98: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-0/wal/wals/05487855a8e0460b8ded8d4d209b39d6
I20260812 11:09:30.782444 20290 ts_tablet_manager.cc:1939] T 05487855a8e0460b8ded8d4d209b39d6 P f41759f922954da9b6d51cbfe3691a98: Deleting consensus metadata
I20260812 11:09:30.782459 20288 ts_tablet_manager.cc:1939] T 05487855a8e0460b8ded8d4d209b39d6 P 936988d8930a425ab67afb31c00c7538: Deleting consensus metadata
I20260812 11:09:30.783197 19362 catalog_manager.cc:4841] TS 936988d8930a425ab67afb31c00c7538 (127.17.44.194:41553): tablet 05487855a8e0460b8ded8d4d209b39d6 (table test-table [id=d7e0c7963e604645bb0b9ad181023775]) successfully deleted
I20260812 11:09:30.783197 19358 catalog_manager.cc:4841] TS f41759f922954da9b6d51cbfe3691a98 (127.17.44.193:38151): tablet 05487855a8e0460b8ded8d4d209b39d6 (table test-table [id=d7e0c7963e604645bb0b9ad181023775]) successfully deleted
I20260812 11:09:30.784587 20287 ts_tablet_manager.cc:1918] T 05487855a8e0460b8ded8d4d209b39d6 P 77e67f84942f4e58a3990fed8e009e85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:09:30.784672 20287 log.cc:1196] T 05487855a8e0460b8ded8d4d209b39d6 P 77e67f84942f4e58a3990fed8e009e85: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/05487855a8e0460b8ded8d4d209b39d6
I20260812 11:09:30.784981 20287 ts_tablet_manager.cc:1939] T 05487855a8e0460b8ded8d4d209b39d6 P 77e67f84942f4e58a3990fed8e009e85: Deleting consensus metadata
I20260812 11:09:30.785647 19358 catalog_manager.cc:4841] TS 77e67f84942f4e58a3990fed8e009e85 (127.17.44.195:45919): tablet 05487855a8e0460b8ded8d4d209b39d6 (table test-table [id=d7e0c7963e604645bb0b9ad181023775]) successfully deleted
Aug 12 11:09:30 dist-test-slave-drl0 krb5kdc[19316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532969, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.44.193@KRBTEST.COM
I20260812 11:09:30.794762 20235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:30.777863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51526 (local address 127.17.44.193:38151)
0812 11:09:30.777984 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:30.777991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:30.779366 (+ 1375us) server_negotiation.cc:407] Connection header received
0812 11:09:30.779804 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:30.779808 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:30.779863 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:30.779918 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:30.784152 (+ 4234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.784951 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.786403 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.786616 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.786703 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:30.790642 (+ 3939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:30.790655 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:30.790673 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:30.790703 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:30.792663 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:30.793160 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:30.793166 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:30.793168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:30.793248 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:30.793541 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:30.793544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:30.793546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:30.793999 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:30.794256 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:30.794459 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 11:09:30.794587 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260812 11:09:30.795419 19693 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51526
I20260812 11:09:30.805449 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:30.797258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60438 (local address 127.17.44.254:42883)
0812 11:09:30.797392 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:30.797396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:30.797599 (+ 203us) server_negotiation.cc:407] Connection header received
0812 11:09:30.797754 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:30.797758 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:30.797811 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:30.797896 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:30.798539 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.799199 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.799980 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:30.800224 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:30.800334 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:30.801810 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:30.801814 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:30.801819 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:30.801851 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:30.803711 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:30.804158 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:30.804162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:30.804164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:30.804250 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:30.804555 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:30.804560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:30.804562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:30.804747 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:30.804895 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:30.805131 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 11:09:30.805259 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260812 11:09:30.805991 19379 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60438
I20260812 11:09:30.807471 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 19498
I20260812 11:09:30.825872 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 19717
I20260812 11:09:30.838750 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 19944
I20260812 11:09:30.849336 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 19335
2026-08-12T11:09:30Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3478 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:09:30 dist-test-slave-drl0 krb5kdc[20299](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:09:30 dist-test-slave-drl0 krb5kdc[20299](info): set up 2 sockets
Aug 12 11:09:30 dist-test-slave-drl0 krb5kdc[20299](info): commencing operation
krb5kdc: starting...
W20260812 11:09:32.158648 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:09:32 dist-test-slave-drl0 krb5kdc[20299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:09:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:09:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:09:32.294539 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:33163
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:43601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:33163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:32.414366 20320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:32.414676 20320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:32.414747 20320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:32.418715 20320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:09:32.418802 20320 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 11:09:32.418835 20320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:32.418886 20320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:32.418921 20320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:09:32.418984 20320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:09:32.424583 20320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:43601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:33163
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:33163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:32.425660 20320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:32.426828 20320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:32.435683 20335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:32.435797 20331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:32.435698 20330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:32.436074 20320 server_base.cc:1034] running on GCE node
I20260812 11:09:32.436625 20320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:32.437727 20320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:32.438910 20320 hybrid_clock.cc:648] HybridClock initialized: now 1786532972438874 us; error 48 us; skew 500 ppm
Aug 12 11:09:32 dist-test-slave-drl0 krb5kdc[20299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532972, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:32.446027 20320 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:09:32.447542 20320 webserver.cc:460] Webserver started at http://127.17.44.254:42673/ using document root <none> and password file <none>
I20260812 11:09:32.448661 20320 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:32.448856 20320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:32.449186 20320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:32.452531 20320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "06548a5ccf064f4bb3a3b9ca51faba19"
format_stamp: "Formatted at 2026-08-12 11:09:32 on dist-test-slave-drl0"
server_key: "1f9f75f120932c13758baca1e6ed090d"
server_key_iv: "505cbfc38db75b34bdb2ab26ba5185bb"
server_key_version: "encryptionkey@0"
I20260812 11:09:32.453259 20320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "06548a5ccf064f4bb3a3b9ca51faba19"
format_stamp: "Formatted at 2026-08-12 11:09:32 on dist-test-slave-drl0"
server_key: "1f9f75f120932c13758baca1e6ed090d"
server_key_iv: "505cbfc38db75b34bdb2ab26ba5185bb"
server_key_version: "encryptionkey@0"
I20260812 11:09:32.457327 20320 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 11:09:32.460044 20355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:32.461508 20320 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:09:32.461705 20320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "06548a5ccf064f4bb3a3b9ca51faba19"
format_stamp: "Formatted at 2026-08-12 11:09:32 on dist-test-slave-drl0"
server_key: "1f9f75f120932c13758baca1e6ed090d"
server_key_iv: "505cbfc38db75b34bdb2ab26ba5185bb"
server_key_version: "encryptionkey@0"
I20260812 11:09:32.461827 20320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:32.491576 20320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:32.495882 20320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:32.496068 20320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:32.505043 20320 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:33163
I20260812 11:09:32.505085 20462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:33163 every 8 connection(s)
I20260812 11:09:32.506073 20320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:09:32.509131 20463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:32.512918 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 20320
I20260812 11:09:32.513273 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:09:32.513844 17587 external_mini_cluster.cc:1428] Setting key 35b55fdb0ab906395fa1868bccc72327
I20260812 11:09:32.515261 20463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19: Bootstrap starting.
I20260812 11:09:32.518078 20463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:32.519160 20463 log.cc:824] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:32.521355 20463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19: No bootstrap required, opened a new log
Aug 12 11:09:32 dist-test-slave-drl0 krb5kdc[20299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:32.526081 20463 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06548a5ccf064f4bb3a3b9ca51faba19" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33163 } }
I20260812 11:09:32.526384 20463 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:32.526437 20463 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06548a5ccf064f4bb3a3b9ca51faba19, State: Initialized, Role: FOLLOWER
I20260812 11:09:32.527093 20463 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06548a5ccf064f4bb3a3b9ca51faba19" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33163 } }
I20260812 11:09:32.527278 20463 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:09:32.527354 20463 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:09:32.527473 20463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:32.528630 20463 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06548a5ccf064f4bb3a3b9ca51faba19" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33163 } }
I20260812 11:09:32.529006 20463 leader_election.cc:302] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06548a5ccf064f4bb3a3b9ca51faba19; no voters:
I20260812 11:09:32.529367 20463 leader_election.cc:288] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:09:32.529628 20468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:32.530025 20468 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19 [term 1 LEADER]: Becoming Leader. State: Replica: 06548a5ccf064f4bb3a3b9ca51faba19, State: Running, Role: LEADER
I20260812 11:09:32.530571 20468 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06548a5ccf064f4bb3a3b9ca51faba19" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33163 } }
I20260812 11:09:32.530993 20463 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:09:32.531471 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:32.515370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60280 (local address 127.17.44.254:33163)
0812 11:09:32.515862 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:32.515898 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:32.515945 (+ 47us) server_negotiation.cc:407] Connection header received
0812 11:09:32.516809 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:32.516832 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:32.517247 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:32.517602 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:32.518329 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:32.519982 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:32.520869 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:32.521549 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:32.521721 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:32.524605 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:32.524612 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:32.524629 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:32.524662 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:32.527190 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:32.528033 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:32.528039 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:32.528045 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:32.528188 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:32.528584 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:32.528588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:32.528589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:32.529119 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:32.529340 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:32.529915 (+ 575us) server_negotiation.cc:299] Negotiation successful
0812 11:09:32.530367 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":122,"threads_started":1}
I20260812 11:09:32.532826 20470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06548a5ccf064f4bb3a3b9ca51faba19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06548a5ccf064f4bb3a3b9ca51faba19" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33163 } } }
I20260812 11:09:32.532984 20470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19 [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:32.533324 20469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06548a5ccf064f4bb3a3b9ca51faba19. Latest consensus state: current_term: 1 leader_uuid: "06548a5ccf064f4bb3a3b9ca51faba19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06548a5ccf064f4bb3a3b9ca51faba19" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33163 } } }
I20260812 11:09:32.533427 20469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19 [sys.catalog]: This master's current role is: LEADER
W20260812 11:09:32.536077 20495 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:09:32.536164 20495 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:09:32.536270 20492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:09:32.537179 20492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:09:32.542219 20492 catalog_manager.cc:1347] Generated new cluster ID: df345080ad5a4adea6e502ba8d4accec
I20260812 11:09:32.542285 20492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:09:32.569545 20492 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:09:32.570833 20492 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:09:32.579270 20492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19: Generated new TSK 0
I20260812 11:09:32.580039 20492 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
I20260812 11:09:32.618307 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:33163
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:43601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:32.745146 20506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:32.745445 20506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:32.745498 20506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:32.750906 20506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:32.751075 20506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:32.751128 20506 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 11:09:32.751143 20506 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 11:09:32.751183 20506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:09:32.759430 20506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:43601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.44.254:33163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:32.761031 20506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:32.762072 20506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:32.772516 20519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:32.772518 20515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:32.773005 20517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:32.773414 20506 server_base.cc:1034] running on GCE node
I20260812 11:09:32.773811 20506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:32.774473 20506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:32.775682 20506 hybrid_clock.cc:648] HybridClock initialized: now 1786532972775647 us; error 43 us; skew 500 ppm
Aug 12 11:09:32 dist-test-slave-drl0 krb5kdc[20299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532972, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:32.778748 20506 init.cc:377] Logged in from keytab as kudu/127.17.44.193@KRBTEST.COM (short username kudu)
I20260812 11:09:32.779665 20506 webserver.cc:460] Webserver started at http://127.17.44.193:45421/ using document root <none> and password file <none>
I20260812 11:09:32.780277 20506 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:32.780383 20506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:32.780627 20506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:32.782689 20506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "779b475921a940629f223a43e4cd3d2c"
format_stamp: "Formatted at 2026-08-12 11:09:32 on dist-test-slave-drl0"
server_key: "68adf1991b0a425fb395065243229b48"
server_key_iv: "40b474577ff50cbe9521e4fea3fd60ec"
server_key_version: "encryptionkey@0"
I20260812 11:09:32.783448 20506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "779b475921a940629f223a43e4cd3d2c"
format_stamp: "Formatted at 2026-08-12 11:09:32 on dist-test-slave-drl0"
server_key: "68adf1991b0a425fb395065243229b48"
server_key_iv: "40b474577ff50cbe9521e4fea3fd60ec"
server_key_version: "encryptionkey@0"
I20260812 11:09:32.787696 20506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 11:09:32.790666 20529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:32.792450 20506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:09:32.792580 20506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "779b475921a940629f223a43e4cd3d2c"
format_stamp: "Formatted at 2026-08-12 11:09:32 on dist-test-slave-drl0"
server_key: "68adf1991b0a425fb395065243229b48"
server_key_iv: "40b474577ff50cbe9521e4fea3fd60ec"
server_key_version: "encryptionkey@0"
I20260812 11:09:32.792714 20506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:32.813642 20506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:32.817355 20506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:32.817521 20506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:32.819092 20506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:32.819146 20506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:32.819188 20506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:32.819242 20506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 11:09:32 dist-test-slave-drl0 krb5kdc[20299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532972, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:32.830377 20506 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:44271
I20260812 11:09:32.830816 20721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:44271 every 8 connection(s)
I20260812 11:09:32.831590 20506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:09:32.835156 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:32.819827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:44017 (local address 127.17.44.254:33163)
0812 11:09:32.820015 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:32.820023 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:32.821370 (+ 1347us) server_negotiation.cc:407] Connection header received
0812 11:09:32.822789 (+ 1419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:32.822792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:32.822857 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:32.823004 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:32.824381 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:32.825008 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:32.826240 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:32.826566 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:32.826679 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:32.830205 (+ 3526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:32.830211 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:32.830214 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:32.830250 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:32.832359 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:32.833188 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:32.833195 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:32.833199 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:32.833314 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:32.833776 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:32.833780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:32.833781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:32.834063 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:32.834193 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:32.834755 (+ 562us) server_negotiation.cc:299] Negotiation successful
0812 11:09:32.834903 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 11:09:32.835816 20544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:32.820670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33163 (local address 127.17.44.193:44017)
0812 11:09:32.821191 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:32.821250 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:32.822532 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:32.823177 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:32.823184 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:32.823684 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:32.824224 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:32.824237 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:32.825140 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:32.825143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:32.826115 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:32.826125 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:32.826736 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:32.826741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:32.826988 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:32.827659 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:32.827677 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:32.829665 (+ 1988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:32.832502 (+ 2837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:32.832510 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:32.832531 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:32.832946 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:32.833456 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:32.833459 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:32.833461 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:32.833627 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:32.834234 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:32.834239 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:32.834475 (+ 236us) client_negotiation.cc:769] Sending connection context
0812 11:09:32.834655 (+ 180us) client_negotiation.cc:240] Negotiation successful
0812 11:09:32.834907 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":118,"threads_started":1}
I20260812 11:09:32.836799 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 20506
I20260812 11:09:32.836936 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:09:32.837252 17587 external_mini_cluster.cc:1428] Setting key 4287dbb33120687599bf2c786908b162
I20260812 11:09:32.837307 20723 heartbeater.cc:347] Connected to a master server at 127.17.44.254:33163
I20260812 11:09:32.837631 20723 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:32.838214 20723 heartbeater.cc:511] Master 127.17.44.254:33163 requested a full tablet report, sending...
I20260812 11:09:32.840233 20381 ts_manager.cc:194] Registered new tserver with Master: 779b475921a940629f223a43e4cd3d2c (127.17.44.193:44271)
I20260812 11:09:32.842067 20381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.193@KRBTEST.COM'} at 127.17.44.193:44017
I20260812 11:09:32.853724 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:32.840168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33163 (local address 127.17.44.193:47437)
0812 11:09:32.840540 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:32.840558 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:32.840655 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:32.843226 (+ 2571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:32.843231 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:32.843480 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:32.843651 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:32.843662 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:32.844837 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:32.844840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:32.845929 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:32.845940 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:32.846649 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:32.846655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:32.846766 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:32.848628 (+ 1862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:32.848640 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:32.848964 (+ 324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:32.851508 (+ 2544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:32.851515 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:32.851520 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:32.851815 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:32.852212 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:32.852217 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:32.852221 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:32.852337 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:32.852923 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:32.852927 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:32.853038 (+ 111us) client_negotiation.cc:769] Sending connection context
0812 11:09:32.853262 (+ 224us) client_negotiation.cc:240] Negotiation successful
0812 11:09:32.853459 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":114,"threads_started":1}
I20260812 11:09:32.854291 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:32.842728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:47437 (local address 127.17.44.254:33163)
0812 11:09:32.842847 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:32.842852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:32.842881 (+ 29us) server_negotiation.cc:407] Connection header received
0812 11:09:32.842921 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:32.842949 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:32.843002 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:32.843089 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:32.843796 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:32.844618 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:32.846083 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:32.846415 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:32.846514 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:32.849121 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:32.849129 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:32.849135 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:32.849170 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:32.851207 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:32.851943 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:32.851946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:32.851947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:32.852066 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:32.852474 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:32.852479 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:32.852481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:32.852758 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:32.852870 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:32.854000 (+ 1130us) server_negotiation.cc:299] Negotiation successful
0812 11:09:32.854121 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
WARNING: no policy specified for kudu/127.17.44.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
I20260812 11:09:32.878738 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:33163
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:43601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:33.005297 20738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:33.005596 20738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:33.005677 20738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:33.009909 20738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:33.010010 20738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:33.010082 20738 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 11:09:33.010126 20738 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 11:09:33.010196 20738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:09:33.015616 20738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:43601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.44.254:33163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:33.016687 20738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:33.017719 20738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:33.029587 20754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:33.029608 20750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:33.029587 20751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:33.030055 20738 server_base.cc:1034] running on GCE node
I20260812 11:09:33.030532 20738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:33.031397 20738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:33.032567 20738 hybrid_clock.cc:648] HybridClock initialized: now 1786532973032552 us; error 38 us; skew 500 ppm
Aug 12 11:09:33 dist-test-slave-drl0 krb5kdc[20299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532973, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:33.035995 20738 init.cc:377] Logged in from keytab as kudu/127.17.44.194@KRBTEST.COM (short username kudu)
I20260812 11:09:33.037051 20738 webserver.cc:460] Webserver started at http://127.17.44.194:40129/ using document root <none> and password file <none>
I20260812 11:09:33.037667 20738 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:33.037758 20738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:33.037977 20738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:33.040186 20738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "d181e17ef5d547bca505e1c5e5bbf543"
format_stamp: "Formatted at 2026-08-12 11:09:33 on dist-test-slave-drl0"
server_key: "0e82ba978827016cb01671a492079f52"
server_key_iv: "ee82494c5aa9e534a05e61f3a0dc9056"
server_key_version: "encryptionkey@0"
I20260812 11:09:33.040778 20738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "d181e17ef5d547bca505e1c5e5bbf543"
format_stamp: "Formatted at 2026-08-12 11:09:33 on dist-test-slave-drl0"
server_key: "0e82ba978827016cb01671a492079f52"
server_key_iv: "ee82494c5aa9e534a05e61f3a0dc9056"
server_key_version: "encryptionkey@0"
I20260812 11:09:33.045379 20738 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 11:09:33.048110 20769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:33.049482 20738 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:09:33.049629 20738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "d181e17ef5d547bca505e1c5e5bbf543"
format_stamp: "Formatted at 2026-08-12 11:09:33 on dist-test-slave-drl0"
server_key: "0e82ba978827016cb01671a492079f52"
server_key_iv: "ee82494c5aa9e534a05e61f3a0dc9056"
server_key_version: "encryptionkey@0"
I20260812 11:09:33.049746 20738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:33.064356 20738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:33.067946 20738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:33.068151 20738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:33.069661 20738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:33.069738 20738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:33.069806 20738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:33.069850 20738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 11:09:33 dist-test-slave-drl0 krb5kdc[20299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532973, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:33.081723 20738 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:35985
I20260812 11:09:33.081807 20960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:35985 every 8 connection(s)
I20260812 11:09:33.082861 20738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:09:33.086313 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 20738
I20260812 11:09:33.086475 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:09:33.086745 17587 external_mini_cluster.cc:1428] Setting key 24a890bda20d2b469a3c5b8eb82db578
I20260812 11:09:33.086879 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.071237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:48807 (local address 127.17.44.254:33163)
0812 11:09:33.071370 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:33.071373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:33.072525 (+ 1152us) server_negotiation.cc:407] Connection header received
0812 11:09:33.073811 (+ 1286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:33.073814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:33.073879 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:33.073991 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:33.076014 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.076690 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.078130 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.078388 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.078511 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.082082 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:33.082087 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:33.082090 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:33.082118 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:33.083973 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:33.084499 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:33.084503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:33.084504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:33.084688 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:33.085168 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:33.085182 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:33.085183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:33.085504 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:33.085600 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:33.086459 (+ 859us) server_negotiation.cc:299] Negotiation successful
0812 11:09:33.086708 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 11:09:33.087920 20783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.071710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33163 (local address 127.17.44.194:48807)
0812 11:09:33.072371 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:33.072418 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:33.073573 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:33.074170 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:33.074176 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:33.074710 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:33.075760 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:33.075784 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.076846 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.076850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:33.077945 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:33.077961 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.078557 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.078561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:33.078781 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.079727 (+ 946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:33.079768 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:33.081886 (+ 2118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:33.084087 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:33.084091 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:33.084102 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:33.084360 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:33.084869 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:33.084871 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:33.084872 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:33.085018 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:33.085696 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:33.085705 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:33.086092 (+ 387us) client_negotiation.cc:769] Sending connection context
0812 11:09:33.086372 (+ 280us) client_negotiation.cc:240] Negotiation successful
0812 11:09:33.086868 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":144,"threads_started":1}
I20260812 11:09:33.089396 20962 heartbeater.cc:347] Connected to a master server at 127.17.44.254:33163
I20260812 11:09:33.089578 20962 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:33.090093 20962 heartbeater.cc:511] Master 127.17.44.254:33163 requested a full tablet report, sending...
I20260812 11:09:33.091190 20381 ts_manager.cc:194] Registered new tserver with Master: d181e17ef5d547bca505e1c5e5bbf543 (127.17.44.194:35985)
I20260812 11:09:33.092141 20381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.194@KRBTEST.COM'} at 127.17.44.194:48807
WARNING: no policy specified for kudu/127.17.44.195@KRBTEST.COM; defaulting to no policy
I20260812 11:09:33.102068 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.092587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33163 (local address 127.17.44.194:42719)
0812 11:09:33.092890 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:33.092912 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:33.093057 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:33.093363 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:33.093369 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:33.093634 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:33.093819 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:33.093832 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.094751 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.094758 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:33.095965 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:33.095983 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.096446 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.096454 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:33.096595 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.097377 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:33.097402 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:33.097921 (+ 519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:33.100336 (+ 2415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:33.100339 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:33.100340 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:33.100540 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:33.100870 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:33.100872 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:33.100874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:33.101025 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:33.101523 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:33.101526 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:33.101608 (+ 82us) client_negotiation.cc:769] Sending connection context
0812 11:09:33.101705 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 11:09:33.101900 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":133,"threads_started":1}
I20260812 11:09:33.102185 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.092488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:42719 (local address 127.17.44.254:33163)
0812 11:09:33.092617 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:33.092622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:33.092994 (+ 372us) server_negotiation.cc:407] Connection header received
0812 11:09:33.093180 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:33.093183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:33.093230 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:33.093318 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:33.093969 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.094587 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.096107 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.096313 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.096402 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.098191 (+ 1789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:33.098195 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:33.098198 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:33.098250 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:33.100217 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:33.100655 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:33.100658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:33.100660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:33.100752 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:33.101152 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:33.101156 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:33.101158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:33.101392 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:33.101489 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:33.101814 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 11:09:33.101917 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
I20260812 11:09:33.123556 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:33163
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:43601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:33.247294 20977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:33.247588 20977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:33.247668 20977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:33.252624 20977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:33.252725 20977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:33.252806 20977 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 11:09:33.252840 20977 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 11:09:33.252919 20977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:09:33.258567 20977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:43601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.44.254:33163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:33.259766 20977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:33.260797 20977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:33.270800 20989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:33.270922 20985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:33.270922 20977 server_base.cc:1034] running on GCE node
W20260812 11:09:33.271099 20986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:33.271569 20977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:33.272197 20977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:33.273545 20977 hybrid_clock.cc:648] HybridClock initialized: now 1786532973273423 us; error 40 us; skew 500 ppm
Aug 12 11:09:33 dist-test-slave-drl0 krb5kdc[20299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532973, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:33.277066 20977 init.cc:377] Logged in from keytab as kudu/127.17.44.195@KRBTEST.COM (short username kudu)
I20260812 11:09:33.278044 20977 webserver.cc:460] Webserver started at http://127.17.44.195:36725/ using document root <none> and password file <none>
I20260812 11:09:33.278645 20977 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:33.278743 20977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:33.279055 20977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:33.281258 20977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "5845f893ab0a4e999b4c30e7746f62dc"
format_stamp: "Formatted at 2026-08-12 11:09:33 on dist-test-slave-drl0"
server_key: "30ff150a6573519a1ff3aca2bc18a0f1"
server_key_iv: "0c06354402709f515eea9732f0a14cec"
server_key_version: "encryptionkey@0"
I20260812 11:09:33.281818 20977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "5845f893ab0a4e999b4c30e7746f62dc"
format_stamp: "Formatted at 2026-08-12 11:09:33 on dist-test-slave-drl0"
server_key: "30ff150a6573519a1ff3aca2bc18a0f1"
server_key_iv: "0c06354402709f515eea9732f0a14cec"
server_key_version: "encryptionkey@0"
I20260812 11:09:33.285499 20977 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260812 11:09:33.288729 21001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:33.290380 20977 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:09:33.290522 20977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "5845f893ab0a4e999b4c30e7746f62dc"
format_stamp: "Formatted at 2026-08-12 11:09:33 on dist-test-slave-drl0"
server_key: "30ff150a6573519a1ff3aca2bc18a0f1"
server_key_iv: "0c06354402709f515eea9732f0a14cec"
server_key_version: "encryptionkey@0"
I20260812 11:09:33.290639 20977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:33.317596 20977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:33.321172 20977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:33.321388 20977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:33.323009 20977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:33.323079 20977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:33.323148 20977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:33.323200 20977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 11:09:33 dist-test-slave-drl0 krb5kdc[20299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532973, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:33.337025 20977 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:41799
I20260812 11:09:33.337122 21193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:41799 every 8 connection(s)
I20260812 11:09:33.338517 20977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:09:33.340003 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.324790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:60875 (local address 127.17.44.254:33163)
0812 11:09:33.324933 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:33.324937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:33.326123 (+ 1186us) server_negotiation.cc:407] Connection header received
0812 11:09:33.327461 (+ 1338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:33.327464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:33.327530 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:33.327633 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:33.328740 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.329539 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.330711 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.331153 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.331285 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.335252 (+ 3967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:33.335256 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:33.335259 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:33.335287 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:33.337390 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:33.338073 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:33.338080 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:33.338082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:33.338197 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:33.338705 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:33.338709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:33.338711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:33.338999 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:33.339120 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:33.339750 (+ 630us) server_negotiation.cc:299] Negotiation successful
0812 11:09:33.339876 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 11:09:33.340752 21017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.325232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33163 (local address 127.17.44.195:60875)
0812 11:09:33.325959 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:33.326014 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:33.327237 (+ 1223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:33.327808 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:33.327815 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:33.328127 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:33.328593 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:33.328606 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.329672 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.329675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:33.330560 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:33.330571 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.331358 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.331364 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:33.331570 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.332493 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:33.332513 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:33.335076 (+ 2563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:33.337510 (+ 2434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:33.337516 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:33.337530 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:33.337803 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:33.338323 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:33.338327 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:33.338329 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:33.338591 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:33.339131 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:33.339138 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:33.339443 (+ 305us) client_negotiation.cc:769] Sending connection context
0812 11:09:33.339649 (+ 206us) client_negotiation.cc:240] Negotiation successful
0812 11:09:33.339907 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":137,"threads_started":1}
I20260812 11:09:33.342021 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 20977
I20260812 11:09:33.342152 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:09:33.342471 17587 external_mini_cluster.cc:1428] Setting key 1ad53f204f597bb035d9868896328adb
I20260812 11:09:33.342799 21195 heartbeater.cc:347] Connected to a master server at 127.17.44.254:33163
I20260812 11:09:33.342994 21195 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:33.343586 21195 heartbeater.cc:511] Master 127.17.44.254:33163 requested a full tablet report, sending...
I20260812 11:09:33.344630 20381 ts_manager.cc:194] Registered new tserver with Master: 5845f893ab0a4e999b4c30e7746f62dc (127.17.44.195:41799)
I20260812 11:09:33.345188 20381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.195@KRBTEST.COM'} at 127.17.44.195:60875
I20260812 11:09:33.353061 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.345327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33163 (local address 127.17.44.195:50109)
0812 11:09:33.345648 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:33.345663 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:33.345764 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:33.346065 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:33.346067 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:33.346254 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:33.346350 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:33.346356 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.347483 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.347488 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:33.348166 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:33.348184 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.348714 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.348718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:33.348814 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.349323 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:33.349335 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:33.349674 (+ 339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:33.351518 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:33.351520 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:33.351522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:33.351742 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:33.352098 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:33.352101 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:33.352102 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:33.352189 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:33.352606 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:33.352609 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:33.352691 (+ 82us) client_negotiation.cc:769] Sending connection context
0812 11:09:33.352785 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 11:09:33.352918 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":126,"threads_started":1}
I20260812 11:09:33.353258 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.345383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:50109 (local address 127.17.44.254:33163)
0812 11:09:33.345524 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:33.345528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:33.345811 (+ 283us) server_negotiation.cc:407] Connection header received
0812 11:09:33.345870 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:33.345873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:33.345920 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:33.346011 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:33.346476 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.347231 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.348358 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.348566 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.348654 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.349832 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:33.349834 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:33.349836 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:33.349857 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:33.351386 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:33.351897 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:33.351900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:33.351901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:33.351978 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:33.352305 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:33.352308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:33.352309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:33.352490 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:33.352577 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:33.352866 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 11:09:33.353034 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 11:09:33.356667 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 11:09:33 dist-test-slave-drl0 krb5kdc[20299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 11:09:33 dist-test-slave-drl0 krb5kdc[20299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:33.385893 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.377069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59404 (local address 127.17.44.254:33163)
0812 11:09:33.377288 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:33.377292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:33.377305 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:09:33.377346 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:33.377348 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:33.377391 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:33.377505 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:33.378189 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.378707 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.379720 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.379948 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.380069 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.382661 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:33.382670 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:33.382672 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:33.382695 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:33.384180 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:33.384613 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:33.384618 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:33.384619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:33.384708 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:33.385015 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:33.385019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:33.385020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:33.385199 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:33.385344 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:33.385586 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 11:09:33.385714 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260812 11:09:33.390024 20381 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:09:33.395247 20381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:09:33.411919 21227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.406621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34834 (local address 127.17.44.193:44271)
0812 11:09:33.407057 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:33.407066 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:33.407090 (+ 24us) server_negotiation.cc:407] Connection header received
0812 11:09:33.407170 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:33.407176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:33.407403 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:33.407563 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:09:33.407858 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.408793 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.410415 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.411182 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.411308 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.411475 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:33.411671 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 11:09:33.411775 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":94,"threads_started":1}
I20260812 11:09:33.411938 21224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.405832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:35985 (local address 127.0.0.1:52814)
0812 11:09:33.406451 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:33.406501 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:33.406637 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:33.407798 (+ 1161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:33.407801 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:33.407822 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:09:33.407915 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:33.407923 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.409255 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.409259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:33.410378 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:33.410389 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.411358 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.411361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:33.411478 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.411510 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 11:09:33.411596 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 11:09:33.411716 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":3,"thread_start_us":104,"threads_started":1}
I20260812 11:09:33.411919 21226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.406014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:44271 (local address 127.0.0.1:34834)
0812 11:09:33.406829 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:33.406844 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:33.406959 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:33.407554 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:33.407560 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:33.407582 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:09:33.407728 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:33.407735 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.408918 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.408923 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:33.410275 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:33.410286 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.411310 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.411313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:33.411419 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.411514 (+ 95us) client_negotiation.cc:769] Sending connection context
0812 11:09:33.411639 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 11:09:33.411720 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":83,"thread_start_us":104,"threads_started":1}
I20260812 11:09:33.412451 21225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.405969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52814 (local address 127.17.44.194:35985)
0812 11:09:33.407201 (+ 1232us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:33.407212 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:33.407230 (+ 18us) server_negotiation.cc:407] Connection header received
0812 11:09:33.407292 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:33.407297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:33.407633 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:33.407816 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:09:33.408039 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.409125 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.410515 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.411224 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.412010 (+ 786us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.412178 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:33.412225 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 11:09:33.412304 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1142,"thread_start_us":103,"threads_started":1}
I20260812 11:09:33.415489 21229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.406735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:41799 (local address 127.0.0.1:33390)
0812 11:09:33.407457 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:33.407471 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:33.407560 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:33.408184 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:33.408186 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:33.408204 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:09:33.408300 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:33.408305 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.409667 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.409671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:33.410815 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:33.410827 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.412434 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.412437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:33.415034 (+ 2597us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.415066 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 11:09:33.415168 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 11:09:33.415253 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"mutex_wait_us":2,"thread_start_us":125,"threads_started":1}
I20260812 11:09:33.415516 21231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.407018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33390 (local address 127.17.44.195:41799)
0812 11:09:33.407449 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:33.407463 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:33.407644 (+ 181us) server_negotiation.cc:407] Connection header received
0812 11:09:33.407718 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:33.407723 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:33.407899 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:33.408051 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:09:33.408425 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.409534 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.410990 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.411744 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.411847 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.412657 (+ 810us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:33.415206 (+ 2549us) server_negotiation.cc:299] Negotiation successful
0812 11:09:33.415302 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":124,"threads_started":1}
I20260812 11:09:33.416061 20854 tablet_service.cc:1467] Processing CreateTablet for tablet b04714d3a63343feab6c246bf143d613 (DEFAULT_TABLE table=test-table [id=1b83174ed4ef4220b20e5971c5722017]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:33.416077 20622 tablet_service.cc:1467] Processing CreateTablet for tablet b04714d3a63343feab6c246bf143d613 (DEFAULT_TABLE table=test-table [id=1b83174ed4ef4220b20e5971c5722017]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:33.417582 20854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b04714d3a63343feab6c246bf143d613. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:33.417579 20622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b04714d3a63343feab6c246bf143d613. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:33.419440 21077 tablet_service.cc:1467] Processing CreateTablet for tablet b04714d3a63343feab6c246bf143d613 (DEFAULT_TABLE table=test-table [id=1b83174ed4ef4220b20e5971c5722017]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:33.420429 21077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b04714d3a63343feab6c246bf143d613. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:33.425987 21233 tablet_bootstrap.cc:492] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543: Bootstrap starting.
I20260812 11:09:33.426097 21232 tablet_bootstrap.cc:492] T b04714d3a63343feab6c246bf143d613 P 779b475921a940629f223a43e4cd3d2c: Bootstrap starting.
I20260812 11:09:33.427982 21234 tablet_bootstrap.cc:492] T b04714d3a63343feab6c246bf143d613 P 5845f893ab0a4e999b4c30e7746f62dc: Bootstrap starting.
I20260812 11:09:33.428310 21232 tablet_bootstrap.cc:654] T b04714d3a63343feab6c246bf143d613 P 779b475921a940629f223a43e4cd3d2c: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:33.428306 21233 tablet_bootstrap.cc:654] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:33.429209 21233 log.cc:824] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:33.429209 21232 log.cc:824] T b04714d3a63343feab6c246bf143d613 P 779b475921a940629f223a43e4cd3d2c: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:33.429806 21234 tablet_bootstrap.cc:654] T b04714d3a63343feab6c246bf143d613 P 5845f893ab0a4e999b4c30e7746f62dc: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:33.430460 21234 log.cc:824] T b04714d3a63343feab6c246bf143d613 P 5845f893ab0a4e999b4c30e7746f62dc: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:33.431272 21233 tablet_bootstrap.cc:492] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543: No bootstrap required, opened a new log
I20260812 11:09:33.431272 21232 tablet_bootstrap.cc:492] T b04714d3a63343feab6c246bf143d613 P 779b475921a940629f223a43e4cd3d2c: No bootstrap required, opened a new log
I20260812 11:09:33.431478 21232 ts_tablet_manager.cc:1397] T b04714d3a63343feab6c246bf143d613 P 779b475921a940629f223a43e4cd3d2c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:09:33.431478 21233 ts_tablet_manager.cc:1397] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:09:33.432603 21234 tablet_bootstrap.cc:492] T b04714d3a63343feab6c246bf143d613 P 5845f893ab0a4e999b4c30e7746f62dc: No bootstrap required, opened a new log
I20260812 11:09:33.432834 21234 ts_tablet_manager.cc:1397] T b04714d3a63343feab6c246bf143d613 P 5845f893ab0a4e999b4c30e7746f62dc: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 11:09:33.436503 21233 raft_consensus.cc:348] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.436811 21233 raft_consensus.cc:374] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:33.436874 21233 raft_consensus.cc:729] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d181e17ef5d547bca505e1c5e5bbf543, State: Initialized, Role: FOLLOWER
I20260812 11:09:33.436985 21232 raft_consensus.cc:348] T b04714d3a63343feab6c246bf143d613 P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.437219 21232 raft_consensus.cc:374] T b04714d3a63343feab6c246bf143d613 P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:33.437284 21232 raft_consensus.cc:729] T b04714d3a63343feab6c246bf143d613 P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 779b475921a940629f223a43e4cd3d2c, State: Initialized, Role: FOLLOWER
I20260812 11:09:33.437142 21234 raft_consensus.cc:348] T b04714d3a63343feab6c246bf143d613 P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.437433 21234 raft_consensus.cc:374] T b04714d3a63343feab6c246bf143d613 P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:33.437381 21233 consensus_queue.cc:260] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.437536 21234 raft_consensus.cc:729] T b04714d3a63343feab6c246bf143d613 P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5845f893ab0a4e999b4c30e7746f62dc, State: Initialized, Role: FOLLOWER
I20260812 11:09:33.437674 21232 consensus_queue.cc:260] T b04714d3a63343feab6c246bf143d613 P 779b475921a940629f223a43e4cd3d2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.437970 21234 consensus_queue.cc:260] T b04714d3a63343feab6c246bf143d613 P 5845f893ab0a4e999b4c30e7746f62dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.438501 21232 ts_tablet_manager.cc:1428] T b04714d3a63343feab6c246bf143d613 P 779b475921a940629f223a43e4cd3d2c: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 11:09:33.438501 21233 ts_tablet_manager.cc:1428] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 11:09:33.438545 20723 heartbeater.cc:503] Master 127.17.44.254:33163 was elected leader, sending a full tablet report...
I20260812 11:09:33.438946 21234 ts_tablet_manager.cc:1428] T b04714d3a63343feab6c246bf143d613 P 5845f893ab0a4e999b4c30e7746f62dc: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 11:09:33.439035 20962 heartbeater.cc:503] Master 127.17.44.254:33163 was elected leader, sending a full tablet report...
I20260812 11:09:33.439260 21195 heartbeater.cc:503] Master 127.17.44.254:33163 was elected leader, sending a full tablet report...
W20260812 11:09:33.584271 20963 tablet.cc:2369] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:09:33.584401 20727 tablet.cc:2369] T b04714d3a63343feab6c246bf143d613 P 779b475921a940629f223a43e4cd3d2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:09:33.591647 21198 tablet.cc:2369] T b04714d3a63343feab6c246bf143d613 P 5845f893ab0a4e999b4c30e7746f62dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:09:33.597393 21240 raft_consensus.cc:482] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:09:33.597584 21240 raft_consensus.cc:504] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.598970 21240 leader_election.cc:288] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 779b475921a940629f223a43e4cd3d2c (127.17.44.193:44271), 5845f893ab0a4e999b4c30e7746f62dc (127.17.44.195:41799)
I20260812 11:09:33.603101 21231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.599369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:35961 (local address 127.17.44.195:41799)
0812 11:09:33.599606 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:33.599611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:33.599627 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:09:33.599781 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:33.599785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:33.599834 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:33.599921 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:09:33.600207 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.600724 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.601956 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.602436 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.602566 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.602617 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:33.602873 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 11:09:33.602980 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260812 11:09:33.603111 21244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.599256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:41799 (local address 127.17.44.194:35961)
0812 11:09:33.599532 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:33.599549 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:33.599659 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:33.599938 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:33.599941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:33.599959 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:09:33.600091 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:33.600104 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.600881 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.600885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:33.601807 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:33.601818 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.602671 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.602674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:33.602745 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.602762 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 11:09:33.602846 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 11:09:33.602939 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":3,"thread_start_us":96,"threads_started":1}
I20260812 11:09:33.603163 21227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.599334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:43775 (local address 127.17.44.193:44271)
0812 11:09:33.599539 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:33.599546 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:33.599744 (+ 198us) server_negotiation.cc:407] Connection header received
0812 11:09:33.599898 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:33.599902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:33.599944 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:33.600060 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:09:33.600336 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.600900 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.601909 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.602331 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.602480 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.602538 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:33.602807 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 11:09:33.602896 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260812 11:09:33.603237 21246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.599293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:44271 (local address 127.17.44.194:43775)
0812 11:09:33.599641 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:33.599652 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:33.599786 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:33.600079 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:33.600082 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:33.600099 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:09:33.600226 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:33.600231 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.601035 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.601039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:33.601783 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:33.601791 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.602480 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.602484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:33.602588 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.602611 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 11:09:33.602799 (+ 188us) client_negotiation.cc:240] Negotiation successful
0812 11:09:33.602964 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":33,"thread_start_us":103,"threads_started":1}
I20260812 11:09:33.603789 21121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04714d3a63343feab6c246bf143d613" candidate_uuid: "d181e17ef5d547bca505e1c5e5bbf543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5845f893ab0a4e999b4c30e7746f62dc" is_pre_election: true
I20260812 11:09:33.603861 20651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04714d3a63343feab6c246bf143d613" candidate_uuid: "d181e17ef5d547bca505e1c5e5bbf543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "779b475921a940629f223a43e4cd3d2c" is_pre_election: true
I20260812 11:09:33.604104 21121 raft_consensus.cc:2454] T b04714d3a63343feab6c246bf143d613 P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d181e17ef5d547bca505e1c5e5bbf543 in term 0.
I20260812 11:09:33.604135 20651 raft_consensus.cc:2454] T b04714d3a63343feab6c246bf143d613 P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d181e17ef5d547bca505e1c5e5bbf543 in term 0.
I20260812 11:09:33.604600 20775 leader_election.cc:302] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5845f893ab0a4e999b4c30e7746f62dc, d181e17ef5d547bca505e1c5e5bbf543; no voters:
I20260812 11:09:33.604884 21240 raft_consensus.cc:2781] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:09:33.604975 21240 raft_consensus.cc:482] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:09:33.605021 21240 raft_consensus.cc:3037] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:33.606207 21240 raft_consensus.cc:504] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.606544 21240 leader_election.cc:288] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543 [CANDIDATE]: Term 1 election: Requested vote from peers 779b475921a940629f223a43e4cd3d2c (127.17.44.193:44271), 5845f893ab0a4e999b4c30e7746f62dc (127.17.44.195:41799)
I20260812 11:09:33.606902 21121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04714d3a63343feab6c246bf143d613" candidate_uuid: "d181e17ef5d547bca505e1c5e5bbf543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5845f893ab0a4e999b4c30e7746f62dc"
I20260812 11:09:33.606920 20651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04714d3a63343feab6c246bf143d613" candidate_uuid: "d181e17ef5d547bca505e1c5e5bbf543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "779b475921a940629f223a43e4cd3d2c"
I20260812 11:09:33.607061 20651 raft_consensus.cc:3037] T b04714d3a63343feab6c246bf143d613 P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:33.607061 21121 raft_consensus.cc:3037] T b04714d3a63343feab6c246bf143d613 P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:33.608417 20651 raft_consensus.cc:2454] T b04714d3a63343feab6c246bf143d613 P 779b475921a940629f223a43e4cd3d2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d181e17ef5d547bca505e1c5e5bbf543 in term 1.
I20260812 11:09:33.608762 21121 raft_consensus.cc:2454] T b04714d3a63343feab6c246bf143d613 P 5845f893ab0a4e999b4c30e7746f62dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d181e17ef5d547bca505e1c5e5bbf543 in term 1.
I20260812 11:09:33.608759 20776 leader_election.cc:302] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 779b475921a940629f223a43e4cd3d2c, d181e17ef5d547bca505e1c5e5bbf543; no voters:
I20260812 11:09:33.608958 21240 raft_consensus.cc:2781] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:33.609155 21240 raft_consensus.cc:686] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543 [term 1 LEADER]: Becoming Leader. State: Replica: d181e17ef5d547bca505e1c5e5bbf543, State: Running, Role: LEADER
I20260812 11:09:33.609527 21240 consensus_queue.cc:237] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.614981 20379 catalog_manager.cc:5495] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543 reported cstate change: term changed from 0 to 1, leader changed from <none> to d181e17ef5d547bca505e1c5e5bbf543 (127.17.44.194). New cstate: current_term: 1 leader_uuid: "d181e17ef5d547bca505e1c5e5bbf543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } health_report { overall_health: UNKNOWN } } }
Aug 12 11:09:33 dist-test-slave-drl0 krb5kdc[20299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532972, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:33.677659 21259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.668757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33163 (local address 127.0.0.1:59416)
0812 11:09:33.669086 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:33.669099 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:33.669203 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:33.669440 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:33.669442 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:33.669826 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:33.669924 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:33.669930 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.670699 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.670702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:33.671274 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:33.671283 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.671769 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.671772 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:33.671840 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.672430 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:33.672442 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:33.673949 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:33.675837 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:33.675849 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:33.675856 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:33.676103 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:33.676389 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:33.676392 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:33.676394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:33.676478 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:33.676986 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:33.676994 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:33.677130 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 11:09:33.677265 (+ 135us) client_negotiation.cc:240] Negotiation successful
0812 11:09:33.677442 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":148,"threads_started":1}
I20260812 11:09:33.677659 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.668778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59416 (local address 127.17.44.254:33163)
0812 11:09:33.668959 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:33.668964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:33.669175 (+ 211us) server_negotiation.cc:407] Connection header received
0812 11:09:33.669281 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:33.669284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:33.669325 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:33.669401 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:33.670062 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.670575 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.671402 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.671619 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.671711 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.674091 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:33.674096 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:33.674104 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:33.674124 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:33.675729 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:33.676205 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:33.676210 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:33.676211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:33.676289 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:33.676576 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:33.676581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:33.676582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:33.676755 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:33.676986 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:33.677329 (+ 343us) server_negotiation.cc:299] Negotiation successful
0812 11:09:33.677508 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":64}
I20260812 11:09:33.682135 20379 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:59416:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 11:09:33.682832 20379 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:09:33.689173 21077 tablet_service.cc:1467] Processing CreateTablet for tablet 166ba27367e14c25b908dca56e3dd3f2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4e1d8f501d3946589937ce9605eaacd1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 11:09:33.689116 20854 tablet_service.cc:1467] Processing CreateTablet for tablet 166ba27367e14c25b908dca56e3dd3f2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4e1d8f501d3946589937ce9605eaacd1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 11:09:33.689182 20622 tablet_service.cc:1467] Processing CreateTablet for tablet 166ba27367e14c25b908dca56e3dd3f2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4e1d8f501d3946589937ce9605eaacd1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 11:09:33.689917 20854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 166ba27367e14c25b908dca56e3dd3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:33.689925 21077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 166ba27367e14c25b908dca56e3dd3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:33.689924 20622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 166ba27367e14c25b908dca56e3dd3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:33.692855 21232 tablet_bootstrap.cc:492] T 166ba27367e14c25b908dca56e3dd3f2 P 779b475921a940629f223a43e4cd3d2c: Bootstrap starting.
I20260812 11:09:33.693833 21232 tablet_bootstrap.cc:654] T 166ba27367e14c25b908dca56e3dd3f2 P 779b475921a940629f223a43e4cd3d2c: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:33.693924 21233 tablet_bootstrap.cc:492] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543: Bootstrap starting.
I20260812 11:09:33.693953 21234 tablet_bootstrap.cc:492] T 166ba27367e14c25b908dca56e3dd3f2 P 5845f893ab0a4e999b4c30e7746f62dc: Bootstrap starting.
I20260812 11:09:33.695098 21234 tablet_bootstrap.cc:654] T 166ba27367e14c25b908dca56e3dd3f2 P 5845f893ab0a4e999b4c30e7746f62dc: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:33.695405 21233 tablet_bootstrap.cc:654] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:33.695595 21232 tablet_bootstrap.cc:492] T 166ba27367e14c25b908dca56e3dd3f2 P 779b475921a940629f223a43e4cd3d2c: No bootstrap required, opened a new log
I20260812 11:09:33.695683 21232 ts_tablet_manager.cc:1397] T 166ba27367e14c25b908dca56e3dd3f2 P 779b475921a940629f223a43e4cd3d2c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 11:09:33.696213 21232 raft_consensus.cc:348] T 166ba27367e14c25b908dca56e3dd3f2 P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.696336 21232 raft_consensus.cc:374] T 166ba27367e14c25b908dca56e3dd3f2 P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:33.696404 21232 raft_consensus.cc:729] T 166ba27367e14c25b908dca56e3dd3f2 P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 779b475921a940629f223a43e4cd3d2c, State: Initialized, Role: FOLLOWER
I20260812 11:09:33.696557 21232 consensus_queue.cc:260] T 166ba27367e14c25b908dca56e3dd3f2 P 779b475921a940629f223a43e4cd3d2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.696869 21232 ts_tablet_manager.cc:1428] T 166ba27367e14c25b908dca56e3dd3f2 P 779b475921a940629f223a43e4cd3d2c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 11:09:33.696981 21234 tablet_bootstrap.cc:492] T 166ba27367e14c25b908dca56e3dd3f2 P 5845f893ab0a4e999b4c30e7746f62dc: No bootstrap required, opened a new log
I20260812 11:09:33.697073 21234 ts_tablet_manager.cc:1397] T 166ba27367e14c25b908dca56e3dd3f2 P 5845f893ab0a4e999b4c30e7746f62dc: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 11:09:33.697153 21233 tablet_bootstrap.cc:492] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543: No bootstrap required, opened a new log
I20260812 11:09:33.697261 21233 ts_tablet_manager.cc:1397] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 11:09:33.697553 21234 raft_consensus.cc:348] T 166ba27367e14c25b908dca56e3dd3f2 P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.697685 21234 raft_consensus.cc:374] T 166ba27367e14c25b908dca56e3dd3f2 P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:33.697733 21234 raft_consensus.cc:729] T 166ba27367e14c25b908dca56e3dd3f2 P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5845f893ab0a4e999b4c30e7746f62dc, State: Initialized, Role: FOLLOWER
I20260812 11:09:33.697878 21234 consensus_queue.cc:260] T 166ba27367e14c25b908dca56e3dd3f2 P 5845f893ab0a4e999b4c30e7746f62dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.697865 21241 tablet_replica.cc:439] T 166ba27367e14c25b908dca56e3dd3f2 P 779b475921a940629f223a43e4cd3d2c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } } }
I20260812 11:09:33.698055 21241 tablet_replica.cc:442] T 166ba27367e14c25b908dca56e3dd3f2 P 779b475921a940629f223a43e4cd3d2c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:09:33.698167 21234 ts_tablet_manager.cc:1428] T 166ba27367e14c25b908dca56e3dd3f2 P 5845f893ab0a4e999b4c30e7746f62dc: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260812 11:09:33.698398 21242 tablet_replica.cc:439] T 166ba27367e14c25b908dca56e3dd3f2 P 5845f893ab0a4e999b4c30e7746f62dc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } } }
I20260812 11:09:33.698544 21242 tablet_replica.cc:442] T 166ba27367e14c25b908dca56e3dd3f2 P 5845f893ab0a4e999b4c30e7746f62dc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:09:33.699174 21233 raft_consensus.cc:348] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.699301 21233 raft_consensus.cc:374] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:33.699338 21233 raft_consensus.cc:729] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d181e17ef5d547bca505e1c5e5bbf543, State: Initialized, Role: FOLLOWER
I20260812 11:09:33.699477 21233 consensus_queue.cc:260] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.699738 21240 tablet_replica.cc:439] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } } }
I20260812 11:09:33.699885 21240 tablet_replica.cc:442] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:09:33.700981 21233 ts_tablet_manager.cc:1428] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260812 11:09:33.719458 21240 raft_consensus.cc:482] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:09:33.719599 21240 raft_consensus.cc:504] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.720260 21240 leader_election.cc:288] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 779b475921a940629f223a43e4cd3d2c (127.17.44.193:44271), 5845f893ab0a4e999b4c30e7746f62dc (127.17.44.195:41799)
I20260812 11:09:33.720737 20651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166ba27367e14c25b908dca56e3dd3f2" candidate_uuid: "d181e17ef5d547bca505e1c5e5bbf543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "779b475921a940629f223a43e4cd3d2c" is_pre_election: true
I20260812 11:09:33.720897 20651 raft_consensus.cc:2454] T 166ba27367e14c25b908dca56e3dd3f2 P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d181e17ef5d547bca505e1c5e5bbf543 in term 0.
I20260812 11:09:33.720928 21121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166ba27367e14c25b908dca56e3dd3f2" candidate_uuid: "d181e17ef5d547bca505e1c5e5bbf543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5845f893ab0a4e999b4c30e7746f62dc" is_pre_election: true
I20260812 11:09:33.721128 21121 raft_consensus.cc:2454] T 166ba27367e14c25b908dca56e3dd3f2 P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d181e17ef5d547bca505e1c5e5bbf543 in term 0.
I20260812 11:09:33.721242 20776 leader_election.cc:302] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 779b475921a940629f223a43e4cd3d2c, d181e17ef5d547bca505e1c5e5bbf543; no voters:
I20260812 11:09:33.721546 21240 raft_consensus.cc:2781] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:09:33.721624 21240 raft_consensus.cc:482] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:09:33.721683 21240 raft_consensus.cc:3037] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:33.722731 21240 raft_consensus.cc:504] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.723096 21240 leader_election.cc:288] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543 [CANDIDATE]: Term 1 election: Requested vote from peers 779b475921a940629f223a43e4cd3d2c (127.17.44.193:44271), 5845f893ab0a4e999b4c30e7746f62dc (127.17.44.195:41799)
I20260812 11:09:33.723553 20651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166ba27367e14c25b908dca56e3dd3f2" candidate_uuid: "d181e17ef5d547bca505e1c5e5bbf543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "779b475921a940629f223a43e4cd3d2c"
I20260812 11:09:33.723564 21121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166ba27367e14c25b908dca56e3dd3f2" candidate_uuid: "d181e17ef5d547bca505e1c5e5bbf543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5845f893ab0a4e999b4c30e7746f62dc"
I20260812 11:09:33.723678 20651 raft_consensus.cc:3037] T 166ba27367e14c25b908dca56e3dd3f2 P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:33.723677 21121 raft_consensus.cc:3037] T 166ba27367e14c25b908dca56e3dd3f2 P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:33.724817 20651 raft_consensus.cc:2454] T 166ba27367e14c25b908dca56e3dd3f2 P 779b475921a940629f223a43e4cd3d2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d181e17ef5d547bca505e1c5e5bbf543 in term 1.
I20260812 11:09:33.724817 21121 raft_consensus.cc:2454] T 166ba27367e14c25b908dca56e3dd3f2 P 5845f893ab0a4e999b4c30e7746f62dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d181e17ef5d547bca505e1c5e5bbf543 in term 1.
I20260812 11:09:33.725152 20775 leader_election.cc:302] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5845f893ab0a4e999b4c30e7746f62dc, d181e17ef5d547bca505e1c5e5bbf543; no voters:
I20260812 11:09:33.725512 21240 raft_consensus.cc:2781] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:33.725703 21240 raft_consensus.cc:686] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543 [term 1 LEADER]: Becoming Leader. State: Replica: d181e17ef5d547bca505e1c5e5bbf543, State: Running, Role: LEADER
I20260812 11:09:33.725939 21240 consensus_queue.cc:237] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.727448 21247 tablet_replica.cc:439] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543: TxnStatusTablet state changed. Reason: New leader d181e17ef5d547bca505e1c5e5bbf543. Latest consensus state: current_term: 1 leader_uuid: "d181e17ef5d547bca505e1c5e5bbf543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } } }
I20260812 11:09:33.727576 21247 tablet_replica.cc:442] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543: This TxnStatusTablet replica's current role is: LEADER
I20260812 11:09:33.727994 21267 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 11:09:33.728304 21267 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 11:09:33.728631 20385 catalog_manager.cc:5495] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543 reported cstate change: term changed from 0 to 1, leader changed from <none> to d181e17ef5d547bca505e1c5e5bbf543 (127.17.44.194). New cstate: current_term: 1 leader_uuid: "d181e17ef5d547bca505e1c5e5bbf543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } health_report { overall_health: UNKNOWN } } }
I20260812 11:09:33.751168 21259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.746713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:35985 (local address 127.0.0.1:52818)
0812 11:09:33.746971 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:33.746990 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:33.747142 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:33.747514 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:33.747516 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:33.747545 (+ 29us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 11:09:33.747660 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:33.747666 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.748787 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.748790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:33.749405 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:33.749413 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.750000 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.750004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:33.750102 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.750146 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 11:09:33.750818 (+ 672us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 11:09:33.750834 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 11:09:33.750921 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 11:09:33.751028 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260812 11:09:33.751168 21225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.746873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52818 (local address 127.17.44.194:35985)
0812 11:09:33.747035 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:33.747041 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:33.747067 (+ 26us) server_negotiation.cc:407] Connection header received
0812 11:09:33.747285 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:33.747288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:33.747353 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:33.747463 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:09:33.747847 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.748658 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.749644 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.749878 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.749996 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.750307 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:09:33.750710 (+ 403us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:09:33.750841 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:33.750966 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 11:09:33.751022 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 11:09:33.756999 20651 raft_consensus.cc:1261] T 166ba27367e14c25b908dca56e3dd3f2 P 779b475921a940629f223a43e4cd3d2c [term 1 FOLLOWER]: Refusing update from remote peer d181e17ef5d547bca505e1c5e5bbf543: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:09:33.756978 21121 raft_consensus.cc:1261] T 166ba27367e14c25b908dca56e3dd3f2 P 5845f893ab0a4e999b4c30e7746f62dc [term 1 FOLLOWER]: Refusing update from remote peer d181e17ef5d547bca505e1c5e5bbf543: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:09:33.758878 21240 consensus_queue.cc:1035] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:09:33.759402 21247 consensus_queue.cc:1035] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:09:33.763975 21241 tablet_replica.cc:439] T 166ba27367e14c25b908dca56e3dd3f2 P 779b475921a940629f223a43e4cd3d2c: TxnStatusTablet state changed. Reason: New leader d181e17ef5d547bca505e1c5e5bbf543. Latest consensus state: current_term: 1 leader_uuid: "d181e17ef5d547bca505e1c5e5bbf543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } } }
I20260812 11:09:33.764127 21241 tablet_replica.cc:442] T 166ba27367e14c25b908dca56e3dd3f2 P 779b475921a940629f223a43e4cd3d2c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:09:33.765419 21242 tablet_replica.cc:439] T 166ba27367e14c25b908dca56e3dd3f2 P 5845f893ab0a4e999b4c30e7746f62dc: TxnStatusTablet state changed. Reason: New leader d181e17ef5d547bca505e1c5e5bbf543. Latest consensus state: current_term: 1 leader_uuid: "d181e17ef5d547bca505e1c5e5bbf543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } } }
I20260812 11:09:33.765558 21242 tablet_replica.cc:442] T 166ba27367e14c25b908dca56e3dd3f2 P 5845f893ab0a4e999b4c30e7746f62dc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:09:33.774892 21242 tablet_replica.cc:439] T 166ba27367e14c25b908dca56e3dd3f2 P 5845f893ab0a4e999b4c30e7746f62dc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d181e17ef5d547bca505e1c5e5bbf543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } } }
I20260812 11:09:33.775045 21242 tablet_replica.cc:442] T 166ba27367e14c25b908dca56e3dd3f2 P 5845f893ab0a4e999b4c30e7746f62dc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:09:33.775059 21247 tablet_replica.cc:439] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d181e17ef5d547bca505e1c5e5bbf543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } } }
I20260812 11:09:33.775162 21247 tablet_replica.cc:442] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543: This TxnStatusTablet replica's current role is: LEADER
I20260812 11:09:33.775416 21280 tablet_replica.cc:439] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d181e17ef5d547bca505e1c5e5bbf543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } } }
I20260812 11:09:33.775537 21280 tablet_replica.cc:442] T 166ba27367e14c25b908dca56e3dd3f2 P d181e17ef5d547bca505e1c5e5bbf543: This TxnStatusTablet replica's current role is: LEADER
I20260812 11:09:33.778694 21275 mvcc.cc:205] Tried to move back new op lower bound from 7317639060501348352 to 7317639060383440896. Current Snapshot: MvccSnapshot[applied={T|T < 7317639060501348352}]
I20260812 11:09:33.780660 21241 tablet_replica.cc:439] T 166ba27367e14c25b908dca56e3dd3f2 P 779b475921a940629f223a43e4cd3d2c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d181e17ef5d547bca505e1c5e5bbf543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } } }
I20260812 11:09:33.780884 21241 tablet_replica.cc:442] T 166ba27367e14c25b908dca56e3dd3f2 P 779b475921a940629f223a43e4cd3d2c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:09:33.781455 21278 mvcc.cc:205] Tried to move back new op lower bound from 7317639060501348352 to 7317639060383440896. Current Snapshot: MvccSnapshot[applied={T|T < 7317639060501348352}]
I20260812 11:09:33.782404 21273 mvcc.cc:205] Tried to move back new op lower bound from 7317639060501348352 to 7317639060383440896. Current Snapshot: MvccSnapshot[applied={T|T < 7317639060501348352 or (T in {7317639060501348352})}]
I20260812 11:09:33.788036 21225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.783625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52830 (local address 127.17.44.194:35985)
0812 11:09:33.783763 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:33.783770 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:33.783785 (+ 15us) server_negotiation.cc:407] Connection header received
0812 11:09:33.783834 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:33.783839 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:33.783886 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:33.783963 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:09:33.784547 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.785321 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.786655 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.787031 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.787135 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.787477 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:09:33.787561 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:09:33.787628 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:33.787824 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 11:09:33.787884 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:09:33.805141 21225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.801535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:59143 (local address 127.17.44.194:35985)
0812 11:09:33.801639 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:33.801645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:33.801663 (+ 18us) server_negotiation.cc:407] Connection header received
0812 11:09:33.801830 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:33.801833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:33.801879 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:33.801963 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:09:33.802299 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.802791 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.803963 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.804189 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.804319 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.804645 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:09:33.804720 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:09:33.804805 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:33.804896 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 11:09:33.804977 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":44}
I20260812 11:09:33.805147 21296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:33.799982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:35985 (local address 127.17.44.194:59143)
0812 11:09:33.801599 (+ 1617us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:33.801614 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:33.801716 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:33.802058 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:33.802061 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:33.802080 (+ 19us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 11:09:33.802186 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:33.802190 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.802901 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.802906 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:33.803837 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:33.803852 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:33.804337 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:33.804341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:33.804447 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:33.804486 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 11:09:33.804808 (+ 322us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 11:09:33.804816 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 11:09:33.804878 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 11:09:33.804990 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1495,"mutex_wait_us":20,"thread_start_us":1403,"threads_started":1}
I20260812 11:09:33.813529 20651 raft_consensus.cc:1261] T b04714d3a63343feab6c246bf143d613 P 779b475921a940629f223a43e4cd3d2c [term 1 FOLLOWER]: Refusing update from remote peer d181e17ef5d547bca505e1c5e5bbf543: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:09:33.813529 21121 raft_consensus.cc:1261] T b04714d3a63343feab6c246bf143d613 P 5845f893ab0a4e999b4c30e7746f62dc [term 1 FOLLOWER]: Refusing update from remote peer d181e17ef5d547bca505e1c5e5bbf543: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:09:33.814066 21240 consensus_queue.cc:1035] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:09:33.814348 21240 consensus_queue.cc:1035] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:09:33.871765 20381 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:09:33.871968 20381 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:09:33.874688 20381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19: Sending DeleteTablet for 3 replicas of tablet b04714d3a63343feab6c246bf143d613
I20260812 11:09:33.875625 21077 tablet_service.cc:1514] Processing DeleteTablet for tablet b04714d3a63343feab6c246bf143d613 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:09:33 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:33390
I20260812 11:09:33.875785 20622 tablet_service.cc:1514] Processing DeleteTablet for tablet b04714d3a63343feab6c246bf143d613 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:09:33 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:34834
I20260812 11:09:33.876281 21304 tablet_replica.cc:330] T b04714d3a63343feab6c246bf143d613 P 779b475921a940629f223a43e4cd3d2c: stopping tablet replica
I20260812 11:09:33.876370 20854 tablet_service.cc:1514] Processing DeleteTablet for tablet b04714d3a63343feab6c246bf143d613 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:09:33 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:52814
I20260812 11:09:33.876334 20381 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 11:09:33.876618 21304 raft_consensus.cc:2229] T b04714d3a63343feab6c246bf143d613 P 779b475921a940629f223a43e4cd3d2c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260812 11:09:33.876754 20381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:09:33.876837 21304 raft_consensus.cc:2258] T b04714d3a63343feab6c246bf143d613 P 779b475921a940629f223a43e4cd3d2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:09:33.877094 21300 tablet_replica.cc:330] T b04714d3a63343feab6c246bf143d613 P 5845f893ab0a4e999b4c30e7746f62dc: stopping tablet replica
I20260812 11:09:33.877429 21300 raft_consensus.cc:2229] T b04714d3a63343feab6c246bf143d613 P 5845f893ab0a4e999b4c30e7746f62dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:09:33.877671 21300 raft_consensus.cc:2258] T b04714d3a63343feab6c246bf143d613 P 5845f893ab0a4e999b4c30e7746f62dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:09:33.878360 21305 tablet_replica.cc:330] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543: stopping tablet replica
I20260812 11:09:33.878605 21305 raft_consensus.cc:2229] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:09:33.878998 21305 raft_consensus.cc:2258] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:09:33.879266 21300 ts_tablet_manager.cc:1905] T b04714d3a63343feab6c246bf143d613 P 5845f893ab0a4e999b4c30e7746f62dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:09:33.880143 21305 ts_tablet_manager.cc:1905] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:09:33.882465 21300 ts_tablet_manager.cc:1918] T b04714d3a63343feab6c246bf143d613 P 5845f893ab0a4e999b4c30e7746f62dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:09:33.882570 21300 log.cc:1196] T b04714d3a63343feab6c246bf143d613 P 5845f893ab0a4e999b4c30e7746f62dc: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/b04714d3a63343feab6c246bf143d613
I20260812 11:09:33.882966 21300 ts_tablet_manager.cc:1939] T b04714d3a63343feab6c246bf143d613 P 5845f893ab0a4e999b4c30e7746f62dc: Deleting consensus metadata
I20260812 11:09:33.883340 21305 ts_tablet_manager.cc:1918] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:09:33.883468 21305 log.cc:1196] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/b04714d3a63343feab6c246bf143d613
I20260812 11:09:33.883581 20622 tablet_service.cc:1467] Processing CreateTablet for tablet 924d259f22214074ba7b18f898b96b47 (DEFAULT_TABLE table=test-table [id=08a0834f1a314f01a9325d0ff5379437]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:33.883805 21304 ts_tablet_manager.cc:1905] T b04714d3a63343feab6c246bf143d613 P 779b475921a940629f223a43e4cd3d2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:09:33.883904 20622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 924d259f22214074ba7b18f898b96b47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:33.883899 21305 ts_tablet_manager.cc:1939] T b04714d3a63343feab6c246bf143d613 P d181e17ef5d547bca505e1c5e5bbf543: Deleting consensus metadata
I20260812 11:09:33.886004 21232 tablet_bootstrap.cc:492] T 924d259f22214074ba7b18f898b96b47 P 779b475921a940629f223a43e4cd3d2c: Bootstrap starting.
I20260812 11:09:33.886214 20854 tablet_service.cc:1467] Processing CreateTablet for tablet 924d259f22214074ba7b18f898b96b47 (DEFAULT_TABLE table=test-table [id=08a0834f1a314f01a9325d0ff5379437]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:33.886445 20854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 924d259f22214074ba7b18f898b96b47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:33.887043 21232 tablet_bootstrap.cc:654] T 924d259f22214074ba7b18f898b96b47 P 779b475921a940629f223a43e4cd3d2c: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:33.887079 21304 ts_tablet_manager.cc:1918] T b04714d3a63343feab6c246bf143d613 P 779b475921a940629f223a43e4cd3d2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:09:33.887167 21304 log.cc:1196] T b04714d3a63343feab6c246bf143d613 P 779b475921a940629f223a43e4cd3d2c: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-0/wal/wals/b04714d3a63343feab6c246bf143d613
I20260812 11:09:33.887490 21304 ts_tablet_manager.cc:1939] T b04714d3a63343feab6c246bf143d613 P 779b475921a940629f223a43e4cd3d2c: Deleting consensus metadata
I20260812 11:09:33.889032 21233 tablet_bootstrap.cc:492] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543: Bootstrap starting.
I20260812 11:09:33.889480 20360 catalog_manager.cc:4841] TS d181e17ef5d547bca505e1c5e5bbf543 (127.17.44.194:35985): tablet b04714d3a63343feab6c246bf143d613 (table test-table [id=1b83174ed4ef4220b20e5971c5722017]) successfully deleted
I20260812 11:09:33.889724 20357 catalog_manager.cc:4841] TS 779b475921a940629f223a43e4cd3d2c (127.17.44.193:44271): tablet b04714d3a63343feab6c246bf143d613 (table test-table [id=1b83174ed4ef4220b20e5971c5722017]) successfully deleted
I20260812 11:09:33.890074 21233 tablet_bootstrap.cc:654] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:33.890169 20356 catalog_manager.cc:4841] TS 5845f893ab0a4e999b4c30e7746f62dc (127.17.44.195:41799): tablet b04714d3a63343feab6c246bf143d613 (table test-table [id=1b83174ed4ef4220b20e5971c5722017]) successfully deleted
I20260812 11:09:33.890540 21077 tablet_service.cc:1467] Processing CreateTablet for tablet 924d259f22214074ba7b18f898b96b47 (DEFAULT_TABLE table=test-table [id=08a0834f1a314f01a9325d0ff5379437]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:33.890800 21077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 924d259f22214074ba7b18f898b96b47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:33.891458 21232 tablet_bootstrap.cc:492] T 924d259f22214074ba7b18f898b96b47 P 779b475921a940629f223a43e4cd3d2c: No bootstrap required, opened a new log
I20260812 11:09:33.891546 21232 ts_tablet_manager.cc:1397] T 924d259f22214074ba7b18f898b96b47 P 779b475921a940629f223a43e4cd3d2c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260812 11:09:33.892006 21232 raft_consensus.cc:348] T 924d259f22214074ba7b18f898b96b47 P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.892134 21232 raft_consensus.cc:374] T 924d259f22214074ba7b18f898b96b47 P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:33.892174 21232 raft_consensus.cc:729] T 924d259f22214074ba7b18f898b96b47 P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 779b475921a940629f223a43e4cd3d2c, State: Initialized, Role: FOLLOWER
I20260812 11:09:33.892310 21232 consensus_queue.cc:260] T 924d259f22214074ba7b18f898b96b47 P 779b475921a940629f223a43e4cd3d2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.892593 21232 ts_tablet_manager.cc:1428] T 924d259f22214074ba7b18f898b96b47 P 779b475921a940629f223a43e4cd3d2c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 11:09:33.893337 21234 tablet_bootstrap.cc:492] T 924d259f22214074ba7b18f898b96b47 P 5845f893ab0a4e999b4c30e7746f62dc: Bootstrap starting.
I20260812 11:09:33.894332 21234 tablet_bootstrap.cc:654] T 924d259f22214074ba7b18f898b96b47 P 5845f893ab0a4e999b4c30e7746f62dc: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:33.894650 21233 tablet_bootstrap.cc:492] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543: No bootstrap required, opened a new log
I20260812 11:09:33.894733 21233 ts_tablet_manager.cc:1397] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260812 11:09:33.895982 21234 tablet_bootstrap.cc:492] T 924d259f22214074ba7b18f898b96b47 P 5845f893ab0a4e999b4c30e7746f62dc: No bootstrap required, opened a new log
I20260812 11:09:33.896068 21234 ts_tablet_manager.cc:1397] T 924d259f22214074ba7b18f898b96b47 P 5845f893ab0a4e999b4c30e7746f62dc: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 11:09:33.896548 21234 raft_consensus.cc:348] T 924d259f22214074ba7b18f898b96b47 P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.896656 21234 raft_consensus.cc:374] T 924d259f22214074ba7b18f898b96b47 P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:33.896697 21234 raft_consensus.cc:729] T 924d259f22214074ba7b18f898b96b47 P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5845f893ab0a4e999b4c30e7746f62dc, State: Initialized, Role: FOLLOWER
I20260812 11:09:33.896823 21234 consensus_queue.cc:260] T 924d259f22214074ba7b18f898b96b47 P 5845f893ab0a4e999b4c30e7746f62dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.897112 21234 ts_tablet_manager.cc:1428] T 924d259f22214074ba7b18f898b96b47 P 5845f893ab0a4e999b4c30e7746f62dc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:09:33.898217 21233 raft_consensus.cc:348] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.898355 21233 raft_consensus.cc:374] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:33.898389 21233 raft_consensus.cc:729] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d181e17ef5d547bca505e1c5e5bbf543, State: Initialized, Role: FOLLOWER
I20260812 11:09:33.898550 21233 consensus_queue.cc:260] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.899479 21233 ts_tablet_manager.cc:1428] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260812 11:09:33.923003 21280 raft_consensus.cc:482] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:09:33.923156 21280 raft_consensus.cc:504] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.923655 21280 leader_election.cc:288] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 779b475921a940629f223a43e4cd3d2c (127.17.44.193:44271), 5845f893ab0a4e999b4c30e7746f62dc (127.17.44.195:41799)
I20260812 11:09:33.924131 21118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "924d259f22214074ba7b18f898b96b47" candidate_uuid: "d181e17ef5d547bca505e1c5e5bbf543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5845f893ab0a4e999b4c30e7746f62dc" is_pre_election: true
I20260812 11:09:33.924129 20651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "924d259f22214074ba7b18f898b96b47" candidate_uuid: "d181e17ef5d547bca505e1c5e5bbf543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "779b475921a940629f223a43e4cd3d2c" is_pre_election: true
I20260812 11:09:33.924286 20651 raft_consensus.cc:2454] T 924d259f22214074ba7b18f898b96b47 P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d181e17ef5d547bca505e1c5e5bbf543 in term 0.
I20260812 11:09:33.924286 21118 raft_consensus.cc:2454] T 924d259f22214074ba7b18f898b96b47 P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d181e17ef5d547bca505e1c5e5bbf543 in term 0.
I20260812 11:09:33.924608 20776 leader_election.cc:302] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5845f893ab0a4e999b4c30e7746f62dc, 779b475921a940629f223a43e4cd3d2c, d181e17ef5d547bca505e1c5e5bbf543; no voters:
I20260812 11:09:33.924821 21280 raft_consensus.cc:2781] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:09:33.924876 21280 raft_consensus.cc:482] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:09:33.924903 21280 raft_consensus.cc:3037] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:33.925902 21280 raft_consensus.cc:504] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.926275 21280 leader_election.cc:288] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543 [CANDIDATE]: Term 1 election: Requested vote from peers 779b475921a940629f223a43e4cd3d2c (127.17.44.193:44271), 5845f893ab0a4e999b4c30e7746f62dc (127.17.44.195:41799)
I20260812 11:09:33.926653 21118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "924d259f22214074ba7b18f898b96b47" candidate_uuid: "d181e17ef5d547bca505e1c5e5bbf543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5845f893ab0a4e999b4c30e7746f62dc"
I20260812 11:09:33.926637 20651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "924d259f22214074ba7b18f898b96b47" candidate_uuid: "d181e17ef5d547bca505e1c5e5bbf543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "779b475921a940629f223a43e4cd3d2c"
I20260812 11:09:33.926784 21118 raft_consensus.cc:3037] T 924d259f22214074ba7b18f898b96b47 P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:33.926795 20651 raft_consensus.cc:3037] T 924d259f22214074ba7b18f898b96b47 P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:33.927938 21118 raft_consensus.cc:2454] T 924d259f22214074ba7b18f898b96b47 P 5845f893ab0a4e999b4c30e7746f62dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d181e17ef5d547bca505e1c5e5bbf543 in term 1.
I20260812 11:09:33.928140 20651 raft_consensus.cc:2454] T 924d259f22214074ba7b18f898b96b47 P 779b475921a940629f223a43e4cd3d2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d181e17ef5d547bca505e1c5e5bbf543 in term 1.
I20260812 11:09:33.928259 20775 leader_election.cc:302] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5845f893ab0a4e999b4c30e7746f62dc, d181e17ef5d547bca505e1c5e5bbf543; no voters:
I20260812 11:09:33.928529 21280 raft_consensus.cc:2781] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:33.928637 21280 raft_consensus.cc:686] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543 [term 1 LEADER]: Becoming Leader. State: Replica: d181e17ef5d547bca505e1c5e5bbf543, State: Running, Role: LEADER
I20260812 11:09:33.928794 21280 consensus_queue.cc:237] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:33.930774 20381 catalog_manager.cc:5495] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543 reported cstate change: term changed from 0 to 1, leader changed from <none> to d181e17ef5d547bca505e1c5e5bbf543 (127.17.44.194). New cstate: current_term: 1 leader_uuid: "d181e17ef5d547bca505e1c5e5bbf543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } health_report { overall_health: UNKNOWN } } }
I20260812 11:09:33.953837 20650 raft_consensus.cc:1261] T 924d259f22214074ba7b18f898b96b47 P 779b475921a940629f223a43e4cd3d2c [term 1 FOLLOWER]: Refusing update from remote peer d181e17ef5d547bca505e1c5e5bbf543: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:09:33.954234 21118 raft_consensus.cc:1261] T 924d259f22214074ba7b18f898b96b47 P 5845f893ab0a4e999b4c30e7746f62dc [term 1 FOLLOWER]: Refusing update from remote peer d181e17ef5d547bca505e1c5e5bbf543: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:09:33.954329 21284 consensus_queue.cc:1035] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:09:33.954700 21280 consensus_queue.cc:1035] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 11:09:33.971273 21273 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 11:09:35.528818 20381 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:09:35.528986 20381 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:09:35.531718 20381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19: Sending DeleteTablet for 3 replicas of tablet 924d259f22214074ba7b18f898b96b47
I20260812 11:09:35.532447 21077 tablet_service.cc:1514] Processing DeleteTablet for tablet 924d259f22214074ba7b18f898b96b47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:09:35 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:33390
I20260812 11:09:35.532433 20622 tablet_service.cc:1514] Processing DeleteTablet for tablet 924d259f22214074ba7b18f898b96b47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:09:35 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:34834
I20260812 11:09:35.532612 20854 tablet_service.cc:1514] Processing DeleteTablet for tablet 924d259f22214074ba7b18f898b96b47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:09:35 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:52814
I20260812 11:09:35.533128 21354 tablet_replica.cc:330] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543: stopping tablet replica
I20260812 11:09:35.533130 21352 tablet_replica.cc:330] T 924d259f22214074ba7b18f898b96b47 P 779b475921a940629f223a43e4cd3d2c: stopping tablet replica
I20260812 11:09:35.533278 21354 raft_consensus.cc:2229] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:09:35.533278 21352 raft_consensus.cc:2229] T 924d259f22214074ba7b18f898b96b47 P 779b475921a940629f223a43e4cd3d2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:09:35.533408 21352 raft_consensus.cc:2258] T 924d259f22214074ba7b18f898b96b47 P 779b475921a940629f223a43e4cd3d2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:09:35.533496 21354 raft_consensus.cc:2258] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:09:35.533666 21353 tablet_replica.cc:330] T 924d259f22214074ba7b18f898b96b47 P 5845f893ab0a4e999b4c30e7746f62dc: stopping tablet replica
I20260812 11:09:35.533824 21353 raft_consensus.cc:2229] T 924d259f22214074ba7b18f898b96b47 P 5845f893ab0a4e999b4c30e7746f62dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:09:35.533972 21353 raft_consensus.cc:2258] T 924d259f22214074ba7b18f898b96b47 P 5845f893ab0a4e999b4c30e7746f62dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:09:35.534183 21354 ts_tablet_manager.cc:1905] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:09:35.534538 21352 ts_tablet_manager.cc:1905] T 924d259f22214074ba7b18f898b96b47 P 779b475921a940629f223a43e4cd3d2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:09:35.538034 21353 ts_tablet_manager.cc:1905] T 924d259f22214074ba7b18f898b96b47 P 5845f893ab0a4e999b4c30e7746f62dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:09:35.539296 21354 ts_tablet_manager.cc:1918] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 11:09:35.539390 21354 log.cc:1196] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/924d259f22214074ba7b18f898b96b47
I20260812 11:09:35.539723 21354 ts_tablet_manager.cc:1939] T 924d259f22214074ba7b18f898b96b47 P d181e17ef5d547bca505e1c5e5bbf543: Deleting consensus metadata
I20260812 11:09:35.539796 21352 ts_tablet_manager.cc:1918] T 924d259f22214074ba7b18f898b96b47 P 779b475921a940629f223a43e4cd3d2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 11:09:35.539870 21352 log.cc:1196] T 924d259f22214074ba7b18f898b96b47 P 779b475921a940629f223a43e4cd3d2c: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-0/wal/wals/924d259f22214074ba7b18f898b96b47
I20260812 11:09:35.540158 21352 ts_tablet_manager.cc:1939] T 924d259f22214074ba7b18f898b96b47 P 779b475921a940629f223a43e4cd3d2c: Deleting consensus metadata
I20260812 11:09:35.540551 20360 catalog_manager.cc:4841] TS d181e17ef5d547bca505e1c5e5bbf543 (127.17.44.194:35985): tablet 924d259f22214074ba7b18f898b96b47 (table test-table [id=08a0834f1a314f01a9325d0ff5379437]) successfully deleted
I20260812 11:09:35.540786 20357 catalog_manager.cc:4841] TS 779b475921a940629f223a43e4cd3d2c (127.17.44.193:44271): tablet 924d259f22214074ba7b18f898b96b47 (table test-table [id=08a0834f1a314f01a9325d0ff5379437]) successfully deleted
I20260812 11:09:35.541178 21353 ts_tablet_manager.cc:1918] T 924d259f22214074ba7b18f898b96b47 P 5845f893ab0a4e999b4c30e7746f62dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 11:09:35.541265 21353 log.cc:1196] T 924d259f22214074ba7b18f898b96b47 P 5845f893ab0a4e999b4c30e7746f62dc: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/924d259f22214074ba7b18f898b96b47
I20260812 11:09:35.541509 21353 ts_tablet_manager.cc:1939] T 924d259f22214074ba7b18f898b96b47 P 5845f893ab0a4e999b4c30e7746f62dc: Deleting consensus metadata
I20260812 11:09:35.542356 20356 catalog_manager.cc:4841] TS 5845f893ab0a4e999b4c30e7746f62dc (127.17.44.195:41799): tablet 924d259f22214074ba7b18f898b96b47 (table test-table [id=08a0834f1a314f01a9325d0ff5379437]) successfully deleted
Aug 12 11:09:35 dist-test-slave-drl0 krb5kdc[20299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532975, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 11:09:35 dist-test-slave-drl0 krb5kdc[20299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532975, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:35.565044 21366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:35.555927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59426 (local address 127.17.44.254:33163)
0812 11:09:35.556224 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:35.556230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:35.556245 (+ 15us) server_negotiation.cc:407] Connection header received
0812 11:09:35.556310 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:35.556312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:35.556354 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:35.556434 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:35.557159 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:35.557722 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:35.558612 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:35.558837 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:35.559132 (+ 295us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:35.561557 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:35.561560 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:35.561562 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:35.561583 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:35.563136 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:35.563571 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:35.563574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:35.563576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:35.563659 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:35.564039 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:35.564042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:35.564044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:35.564225 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:35.564336 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:35.564656 (+ 320us) server_negotiation.cc:299] Negotiation successful
0812 11:09:35.564824 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":115,"threads_started":1}
I20260812 11:09:35.568163 20381 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59426:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:09:35.568639 20381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:09:35.574048 20854 tablet_service.cc:1467] Processing CreateTablet for tablet ee15280deaa04a9f82ef6217a930526d (DEFAULT_TABLE table=test-table [id=68068158f9fb44f19b750c923a674528]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:35.574041 20622 tablet_service.cc:1467] Processing CreateTablet for tablet ee15280deaa04a9f82ef6217a930526d (DEFAULT_TABLE table=test-table [id=68068158f9fb44f19b750c923a674528]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:35.574342 20622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee15280deaa04a9f82ef6217a930526d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:35.574342 20854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee15280deaa04a9f82ef6217a930526d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:35.574353 21077 tablet_service.cc:1467] Processing CreateTablet for tablet ee15280deaa04a9f82ef6217a930526d (DEFAULT_TABLE table=test-table [id=68068158f9fb44f19b750c923a674528]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:35.574628 21077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee15280deaa04a9f82ef6217a930526d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:35.577046 21367 tablet_bootstrap.cc:492] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543: Bootstrap starting.
I20260812 11:09:35.577256 21370 tablet_bootstrap.cc:492] T ee15280deaa04a9f82ef6217a930526d P 5845f893ab0a4e999b4c30e7746f62dc: Bootstrap starting.
I20260812 11:09:35.577915 21369 tablet_bootstrap.cc:492] T ee15280deaa04a9f82ef6217a930526d P 779b475921a940629f223a43e4cd3d2c: Bootstrap starting.
I20260812 11:09:35.578275 21367 tablet_bootstrap.cc:654] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:35.578467 21370 tablet_bootstrap.cc:654] T ee15280deaa04a9f82ef6217a930526d P 5845f893ab0a4e999b4c30e7746f62dc: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:35.579262 21369 tablet_bootstrap.cc:654] T ee15280deaa04a9f82ef6217a930526d P 779b475921a940629f223a43e4cd3d2c: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:35.579602 21367 tablet_bootstrap.cc:492] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543: No bootstrap required, opened a new log
I20260812 11:09:35.579705 21367 ts_tablet_manager.cc:1397] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 11:09:35.580212 21367 raft_consensus.cc:348] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:35.580310 21367 raft_consensus.cc:374] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:35.580333 21367 raft_consensus.cc:729] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d181e17ef5d547bca505e1c5e5bbf543, State: Initialized, Role: FOLLOWER
I20260812 11:09:35.580466 21370 tablet_bootstrap.cc:492] T ee15280deaa04a9f82ef6217a930526d P 5845f893ab0a4e999b4c30e7746f62dc: No bootstrap required, opened a new log
I20260812 11:09:35.580475 21367 consensus_queue.cc:260] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:35.580559 21370 ts_tablet_manager.cc:1397] T ee15280deaa04a9f82ef6217a930526d P 5845f893ab0a4e999b4c30e7746f62dc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 11:09:35.580753 21367 ts_tablet_manager.cc:1428] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:09:35.581074 21370 raft_consensus.cc:348] T ee15280deaa04a9f82ef6217a930526d P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:35.581200 21370 raft_consensus.cc:374] T ee15280deaa04a9f82ef6217a930526d P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:35.581254 21370 raft_consensus.cc:729] T ee15280deaa04a9f82ef6217a930526d P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5845f893ab0a4e999b4c30e7746f62dc, State: Initialized, Role: FOLLOWER
I20260812 11:09:35.581387 21370 consensus_queue.cc:260] T ee15280deaa04a9f82ef6217a930526d P 5845f893ab0a4e999b4c30e7746f62dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:35.581885 21370 ts_tablet_manager.cc:1428] T ee15280deaa04a9f82ef6217a930526d P 5845f893ab0a4e999b4c30e7746f62dc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:09:35.582624 21369 tablet_bootstrap.cc:492] T ee15280deaa04a9f82ef6217a930526d P 779b475921a940629f223a43e4cd3d2c: No bootstrap required, opened a new log
I20260812 11:09:35.582720 21369 ts_tablet_manager.cc:1397] T ee15280deaa04a9f82ef6217a930526d P 779b475921a940629f223a43e4cd3d2c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 11:09:35.583225 21369 raft_consensus.cc:348] T ee15280deaa04a9f82ef6217a930526d P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:35.583372 21369 raft_consensus.cc:374] T ee15280deaa04a9f82ef6217a930526d P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:35.583408 21369 raft_consensus.cc:729] T ee15280deaa04a9f82ef6217a930526d P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 779b475921a940629f223a43e4cd3d2c, State: Initialized, Role: FOLLOWER
I20260812 11:09:35.583552 21369 consensus_queue.cc:260] T ee15280deaa04a9f82ef6217a930526d P 779b475921a940629f223a43e4cd3d2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:35.583987 21369 ts_tablet_manager.cc:1428] T ee15280deaa04a9f82ef6217a930526d P 779b475921a940629f223a43e4cd3d2c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 11:09:35.660104 21329 raft_consensus.cc:482] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:09:35.660281 21329 raft_consensus.cc:504] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:35.660708 21329 leader_election.cc:288] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 779b475921a940629f223a43e4cd3d2c (127.17.44.193:44271), 5845f893ab0a4e999b4c30e7746f62dc (127.17.44.195:41799)
I20260812 11:09:35.661211 21121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee15280deaa04a9f82ef6217a930526d" candidate_uuid: "d181e17ef5d547bca505e1c5e5bbf543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5845f893ab0a4e999b4c30e7746f62dc" is_pre_election: true
I20260812 11:09:35.661217 20650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee15280deaa04a9f82ef6217a930526d" candidate_uuid: "d181e17ef5d547bca505e1c5e5bbf543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "779b475921a940629f223a43e4cd3d2c" is_pre_election: true
I20260812 11:09:35.661369 21121 raft_consensus.cc:2454] T ee15280deaa04a9f82ef6217a930526d P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d181e17ef5d547bca505e1c5e5bbf543 in term 0.
I20260812 11:09:35.661386 20650 raft_consensus.cc:2454] T ee15280deaa04a9f82ef6217a930526d P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d181e17ef5d547bca505e1c5e5bbf543 in term 0.
I20260812 11:09:35.661715 20775 leader_election.cc:302] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5845f893ab0a4e999b4c30e7746f62dc, d181e17ef5d547bca505e1c5e5bbf543; no voters:
I20260812 11:09:35.661890 21329 raft_consensus.cc:2781] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:09:35.661957 21329 raft_consensus.cc:482] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:09:35.662007 21329 raft_consensus.cc:3037] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:35.663082 21329 raft_consensus.cc:504] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:35.663434 21329 leader_election.cc:288] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543 [CANDIDATE]: Term 1 election: Requested vote from peers 779b475921a940629f223a43e4cd3d2c (127.17.44.193:44271), 5845f893ab0a4e999b4c30e7746f62dc (127.17.44.195:41799)
I20260812 11:09:35.663841 21121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee15280deaa04a9f82ef6217a930526d" candidate_uuid: "d181e17ef5d547bca505e1c5e5bbf543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5845f893ab0a4e999b4c30e7746f62dc"
I20260812 11:09:35.663825 20650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee15280deaa04a9f82ef6217a930526d" candidate_uuid: "d181e17ef5d547bca505e1c5e5bbf543" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "779b475921a940629f223a43e4cd3d2c"
I20260812 11:09:35.663965 20650 raft_consensus.cc:3037] T ee15280deaa04a9f82ef6217a930526d P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:35.663960 21121 raft_consensus.cc:3037] T ee15280deaa04a9f82ef6217a930526d P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:35.665212 21121 raft_consensus.cc:2454] T ee15280deaa04a9f82ef6217a930526d P 5845f893ab0a4e999b4c30e7746f62dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d181e17ef5d547bca505e1c5e5bbf543 in term 1.
I20260812 11:09:35.665266 20650 raft_consensus.cc:2454] T ee15280deaa04a9f82ef6217a930526d P 779b475921a940629f223a43e4cd3d2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d181e17ef5d547bca505e1c5e5bbf543 in term 1.
I20260812 11:09:35.665685 20776 leader_election.cc:302] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5845f893ab0a4e999b4c30e7746f62dc, 779b475921a940629f223a43e4cd3d2c, d181e17ef5d547bca505e1c5e5bbf543; no voters:
I20260812 11:09:35.665953 21329 raft_consensus.cc:2781] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:35.666064 21329 raft_consensus.cc:686] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543 [term 1 LEADER]: Becoming Leader. State: Replica: d181e17ef5d547bca505e1c5e5bbf543, State: Running, Role: LEADER
I20260812 11:09:35.666225 21329 consensus_queue.cc:237] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:35.668337 20381 catalog_manager.cc:5495] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543 reported cstate change: term changed from 0 to 1, leader changed from <none> to d181e17ef5d547bca505e1c5e5bbf543 (127.17.44.194). New cstate: current_term: 1 leader_uuid: "d181e17ef5d547bca505e1c5e5bbf543" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } health_report { overall_health: UNKNOWN } } }
I20260812 11:09:35.691095 21383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:35.687718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52834 (local address 127.17.44.194:35985)
0812 11:09:35.688011 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:35.688020 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:35.688031 (+ 11us) server_negotiation.cc:407] Connection header received
0812 11:09:35.688066 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:35.688069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:35.688108 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:35.688196 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:09:35.688459 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:35.689021 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:35.689897 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:35.690181 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:35.690293 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:35.690554 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:09:35.690631 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:09:35.690706 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:35.690842 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 11:09:35.690914 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":103,"threads_started":1}
I20260812 11:09:35.698820 20650 raft_consensus.cc:1261] T ee15280deaa04a9f82ef6217a930526d P 779b475921a940629f223a43e4cd3d2c [term 1 FOLLOWER]: Refusing update from remote peer d181e17ef5d547bca505e1c5e5bbf543: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:09:35.698998 21121 raft_consensus.cc:1261] T ee15280deaa04a9f82ef6217a930526d P 5845f893ab0a4e999b4c30e7746f62dc [term 1 FOLLOWER]: Refusing update from remote peer d181e17ef5d547bca505e1c5e5bbf543: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:09:35.699251 21329 consensus_queue.cc:1035] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:09:35.699414 21333 consensus_queue.cc:1035] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:09:35.733230 20379 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:09:35.733434 20379 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:09:35.736339 20379 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19: Sending DeleteTablet for 3 replicas of tablet ee15280deaa04a9f82ef6217a930526d
I20260812 11:09:35.737082 20622 tablet_service.cc:1514] Processing DeleteTablet for tablet ee15280deaa04a9f82ef6217a930526d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:09:35 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:34834
I20260812 11:09:35.737231 20854 tablet_service.cc:1514] Processing DeleteTablet for tablet ee15280deaa04a9f82ef6217a930526d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:09:35 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:52814
I20260812 11:09:35.737433 21354 tablet_replica.cc:330] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543: stopping tablet replica
I20260812 11:09:35.737473 21077 tablet_service.cc:1514] Processing DeleteTablet for tablet ee15280deaa04a9f82ef6217a930526d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:09:35 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:33390
I20260812 11:09:35.737561 21354 raft_consensus.cc:2229] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:09:35.737686 21352 tablet_replica.cc:330] T ee15280deaa04a9f82ef6217a930526d P 779b475921a940629f223a43e4cd3d2c: stopping tablet replica
I20260812 11:09:35.737775 21354 raft_consensus.cc:2258] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:09:35.737926 21353 tablet_replica.cc:330] T ee15280deaa04a9f82ef6217a930526d P 5845f893ab0a4e999b4c30e7746f62dc: stopping tablet replica
I20260812 11:09:35.738046 21353 raft_consensus.cc:2229] T ee15280deaa04a9f82ef6217a930526d P 5845f893ab0a4e999b4c30e7746f62dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:09:35.738185 21353 raft_consensus.cc:2258] T ee15280deaa04a9f82ef6217a930526d P 5845f893ab0a4e999b4c30e7746f62dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:09:35.738274 20379 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59426:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 11:09:35.738500 21352 raft_consensus.cc:2229] T ee15280deaa04a9f82ef6217a930526d P 779b475921a940629f223a43e4cd3d2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:09:35.738586 21354 ts_tablet_manager.cc:1905] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:09:35.738626 21352 raft_consensus.cc:2258] T ee15280deaa04a9f82ef6217a930526d P 779b475921a940629f223a43e4cd3d2c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 11:09:35.738693 20379 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:09:35.739043 21353 ts_tablet_manager.cc:1905] T ee15280deaa04a9f82ef6217a930526d P 5845f893ab0a4e999b4c30e7746f62dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:09:35.739359 21352 ts_tablet_manager.cc:1905] T ee15280deaa04a9f82ef6217a930526d P 779b475921a940629f223a43e4cd3d2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:09:35.741357 21354 ts_tablet_manager.cc:1918] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:09:35.741436 21354 log.cc:1196] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/ee15280deaa04a9f82ef6217a930526d
I20260812 11:09:35.741542 21353 ts_tablet_manager.cc:1918] T ee15280deaa04a9f82ef6217a930526d P 5845f893ab0a4e999b4c30e7746f62dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:09:35.741567 21352 ts_tablet_manager.cc:1918] T ee15280deaa04a9f82ef6217a930526d P 779b475921a940629f223a43e4cd3d2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:09:35.741609 21353 log.cc:1196] T ee15280deaa04a9f82ef6217a930526d P 5845f893ab0a4e999b4c30e7746f62dc: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/ee15280deaa04a9f82ef6217a930526d
I20260812 11:09:35.741638 21352 log.cc:1196] T ee15280deaa04a9f82ef6217a930526d P 779b475921a940629f223a43e4cd3d2c: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-0/wal/wals/ee15280deaa04a9f82ef6217a930526d
I20260812 11:09:35.741714 21354 ts_tablet_manager.cc:1939] T ee15280deaa04a9f82ef6217a930526d P d181e17ef5d547bca505e1c5e5bbf543: Deleting consensus metadata
I20260812 11:09:35.741900 21353 ts_tablet_manager.cc:1939] T ee15280deaa04a9f82ef6217a930526d P 5845f893ab0a4e999b4c30e7746f62dc: Deleting consensus metadata
I20260812 11:09:35.741923 21352 ts_tablet_manager.cc:1939] T ee15280deaa04a9f82ef6217a930526d P 779b475921a940629f223a43e4cd3d2c: Deleting consensus metadata
I20260812 11:09:35.742403 20360 catalog_manager.cc:4841] TS d181e17ef5d547bca505e1c5e5bbf543 (127.17.44.194:35985): tablet ee15280deaa04a9f82ef6217a930526d (table test-table [id=68068158f9fb44f19b750c923a674528]) successfully deleted
I20260812 11:09:35.742899 20357 catalog_manager.cc:4841] TS 779b475921a940629f223a43e4cd3d2c (127.17.44.193:44271): tablet ee15280deaa04a9f82ef6217a930526d (table test-table [id=68068158f9fb44f19b750c923a674528]) successfully deleted
I20260812 11:09:35.743403 20356 catalog_manager.cc:4841] TS 5845f893ab0a4e999b4c30e7746f62dc (127.17.44.195:41799): tablet ee15280deaa04a9f82ef6217a930526d (table test-table [id=68068158f9fb44f19b750c923a674528]) successfully deleted
I20260812 11:09:35.746356 20854 tablet_service.cc:1467] Processing CreateTablet for tablet b7db81543a724175be0367508d05f5b9 (DEFAULT_TABLE table=test-table [id=9c41f16f18d547ee888a0d1339626212]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:35.746356 20622 tablet_service.cc:1467] Processing CreateTablet for tablet b7db81543a724175be0367508d05f5b9 (DEFAULT_TABLE table=test-table [id=9c41f16f18d547ee888a0d1339626212]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:35.746405 21077 tablet_service.cc:1467] Processing CreateTablet for tablet b7db81543a724175be0367508d05f5b9 (DEFAULT_TABLE table=test-table [id=9c41f16f18d547ee888a0d1339626212]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:35.746614 20622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7db81543a724175be0367508d05f5b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:35.746614 20854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7db81543a724175be0367508d05f5b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:35.746673 21077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7db81543a724175be0367508d05f5b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:35.748559 21370 tablet_bootstrap.cc:492] T b7db81543a724175be0367508d05f5b9 P 5845f893ab0a4e999b4c30e7746f62dc: Bootstrap starting.
I20260812 11:09:35.748907 21367 tablet_bootstrap.cc:492] T b7db81543a724175be0367508d05f5b9 P d181e17ef5d547bca505e1c5e5bbf543: Bootstrap starting.
I20260812 11:09:35.749189 21369 tablet_bootstrap.cc:492] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c: Bootstrap starting.
I20260812 11:09:35.749914 21370 tablet_bootstrap.cc:654] T b7db81543a724175be0367508d05f5b9 P 5845f893ab0a4e999b4c30e7746f62dc: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:35.749950 21369 tablet_bootstrap.cc:654] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:35.750164 21367 tablet_bootstrap.cc:654] T b7db81543a724175be0367508d05f5b9 P d181e17ef5d547bca505e1c5e5bbf543: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:35.751170 21369 tablet_bootstrap.cc:492] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c: No bootstrap required, opened a new log
I20260812 11:09:35.751492 21369 ts_tablet_manager.cc:1397] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 11:09:35.751819 21367 tablet_bootstrap.cc:492] T b7db81543a724175be0367508d05f5b9 P d181e17ef5d547bca505e1c5e5bbf543: No bootstrap required, opened a new log
I20260812 11:09:35.751894 21367 ts_tablet_manager.cc:1397] T b7db81543a724175be0367508d05f5b9 P d181e17ef5d547bca505e1c5e5bbf543: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 11:09:35.751974 21369 raft_consensus.cc:348] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:35.752076 21369 raft_consensus.cc:374] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:35.752108 21369 raft_consensus.cc:729] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 779b475921a940629f223a43e4cd3d2c, State: Initialized, Role: FOLLOWER
I20260812 11:09:35.752250 21369 consensus_queue.cc:260] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:35.752282 21367 raft_consensus.cc:348] T b7db81543a724175be0367508d05f5b9 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:35.752374 21367 raft_consensus.cc:374] T b7db81543a724175be0367508d05f5b9 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:35.752405 21367 raft_consensus.cc:729] T b7db81543a724175be0367508d05f5b9 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d181e17ef5d547bca505e1c5e5bbf543, State: Initialized, Role: FOLLOWER
I20260812 11:09:35.752658 21369 ts_tablet_manager.cc:1428] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 11:09:35.752766 21367 consensus_queue.cc:260] T b7db81543a724175be0367508d05f5b9 P d181e17ef5d547bca505e1c5e5bbf543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:35.753168 21367 ts_tablet_manager.cc:1428] T b7db81543a724175be0367508d05f5b9 P d181e17ef5d547bca505e1c5e5bbf543: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:09:35.753371 21370 tablet_bootstrap.cc:492] T b7db81543a724175be0367508d05f5b9 P 5845f893ab0a4e999b4c30e7746f62dc: No bootstrap required, opened a new log
I20260812 11:09:35.753465 21370 ts_tablet_manager.cc:1397] T b7db81543a724175be0367508d05f5b9 P 5845f893ab0a4e999b4c30e7746f62dc: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 11:09:35.753973 21370 raft_consensus.cc:348] T b7db81543a724175be0367508d05f5b9 P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:35.754113 21370 raft_consensus.cc:374] T b7db81543a724175be0367508d05f5b9 P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:35.754151 21370 raft_consensus.cc:729] T b7db81543a724175be0367508d05f5b9 P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5845f893ab0a4e999b4c30e7746f62dc, State: Initialized, Role: FOLLOWER
I20260812 11:09:35.754261 21370 consensus_queue.cc:260] T b7db81543a724175be0367508d05f5b9 P 5845f893ab0a4e999b4c30e7746f62dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:35.754563 21370 ts_tablet_manager.cc:1428] T b7db81543a724175be0367508d05f5b9 P 5845f893ab0a4e999b4c30e7746f62dc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:09:35.999549 21377 raft_consensus.cc:482] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:09:35.999716 21377 raft_consensus.cc:504] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:36.001008 21377 leader_election.cc:288] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d181e17ef5d547bca505e1c5e5bbf543 (127.17.44.194:35985), 5845f893ab0a4e999b4c30e7746f62dc (127.17.44.195:41799)
I20260812 11:09:36.005096 21383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:36.001348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:58123 (local address 127.17.44.194:35985)
0812 11:09:36.001513 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:36.001522 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:36.001654 (+ 132us) server_negotiation.cc:407] Connection header received
0812 11:09:36.001902 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:36.001909 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:36.001962 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:36.002023 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:09:36.002345 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:36.003053 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:36.004115 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:36.004526 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:36.004651 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:36.004710 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:36.004897 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 11:09:36.004975 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 11:09:36.005103 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:36.001544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:57239 (local address 127.17.44.195:41799)
0812 11:09:36.001804 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:36.001822 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:36.001839 (+ 17us) server_negotiation.cc:407] Connection header received
0812 11:09:36.001925 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:36.001927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:36.001966 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:36.002061 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:09:36.002391 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:36.002911 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:36.004047 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:36.004496 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:36.004632 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:36.004690 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:36.004883 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 11:09:36.004950 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":84,"threads_started":1}
I20260812 11:09:36.005141 21395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:36.001217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:35985 (local address 127.17.44.193:58123)
0812 11:09:36.001548 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:36.001564 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:36.001749 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:36.002059 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:36.002062 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:36.002079 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:09:36.002232 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:36.002238 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:36.003209 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:36.003213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:36.003978 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:36.003987 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:36.004647 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:36.004651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:36.004766 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:36.004785 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 11:09:36.004877 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 11:09:36.004949 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":4,"spinlock_wait_cycles":7424,"thread_start_us":114,"threads_started":1}
I20260812 11:09:36.005335 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:36.001391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:41799 (local address 127.17.44.193:57239)
0812 11:09:36.001694 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:36.001707 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:36.001801 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:36.002086 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:36.002089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:36.002131 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:09:36.002256 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:36.002261 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:36.003066 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:36.003071 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:36.003920 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:36.003929 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:36.004632 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:36.004636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:36.004738 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:36.004769 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 11:09:36.004865 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 11:09:36.004941 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":3,"spinlock_wait_cycles":768,"thread_start_us":147,"threads_started":1}
I20260812 11:09:36.006000 20868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7db81543a724175be0367508d05f5b9" candidate_uuid: "779b475921a940629f223a43e4cd3d2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d181e17ef5d547bca505e1c5e5bbf543" is_pre_election: true
I20260812 11:09:36.006278 21118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7db81543a724175be0367508d05f5b9" candidate_uuid: "779b475921a940629f223a43e4cd3d2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5845f893ab0a4e999b4c30e7746f62dc" is_pre_election: true
I20260812 11:09:36.006291 20868 raft_consensus.cc:2454] T b7db81543a724175be0367508d05f5b9 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 779b475921a940629f223a43e4cd3d2c in term 0.
I20260812 11:09:36.006412 21118 raft_consensus.cc:2454] T b7db81543a724175be0367508d05f5b9 P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 779b475921a940629f223a43e4cd3d2c in term 0.
I20260812 11:09:36.006803 20530 leader_election.cc:302] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5845f893ab0a4e999b4c30e7746f62dc, 779b475921a940629f223a43e4cd3d2c, d181e17ef5d547bca505e1c5e5bbf543; no voters:
I20260812 11:09:36.007166 21377 raft_consensus.cc:2781] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:09:36.007237 21377 raft_consensus.cc:482] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:09:36.007277 21377 raft_consensus.cc:3037] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:36.008322 21377 raft_consensus.cc:504] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:36.008661 21377 leader_election.cc:288] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c [CANDIDATE]: Term 1 election: Requested vote from peers d181e17ef5d547bca505e1c5e5bbf543 (127.17.44.194:35985), 5845f893ab0a4e999b4c30e7746f62dc (127.17.44.195:41799)
I20260812 11:09:36.009150 21118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7db81543a724175be0367508d05f5b9" candidate_uuid: "779b475921a940629f223a43e4cd3d2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5845f893ab0a4e999b4c30e7746f62dc"
I20260812 11:09:36.009164 20868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7db81543a724175be0367508d05f5b9" candidate_uuid: "779b475921a940629f223a43e4cd3d2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d181e17ef5d547bca505e1c5e5bbf543"
I20260812 11:09:36.009286 21118 raft_consensus.cc:3037] T b7db81543a724175be0367508d05f5b9 P 5845f893ab0a4e999b4c30e7746f62dc [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:36.009286 20868 raft_consensus.cc:3037] T b7db81543a724175be0367508d05f5b9 P d181e17ef5d547bca505e1c5e5bbf543 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:36.010591 20868 raft_consensus.cc:2454] T b7db81543a724175be0367508d05f5b9 P d181e17ef5d547bca505e1c5e5bbf543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 779b475921a940629f223a43e4cd3d2c in term 1.
I20260812 11:09:36.010579 21118 raft_consensus.cc:2454] T b7db81543a724175be0367508d05f5b9 P 5845f893ab0a4e999b4c30e7746f62dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 779b475921a940629f223a43e4cd3d2c in term 1.
I20260812 11:09:36.011221 20530 leader_election.cc:302] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5845f893ab0a4e999b4c30e7746f62dc, 779b475921a940629f223a43e4cd3d2c; no voters:
I20260812 11:09:36.011468 21377 raft_consensus.cc:2781] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:36.011783 21377 raft_consensus.cc:686] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c [term 1 LEADER]: Becoming Leader. State: Replica: 779b475921a940629f223a43e4cd3d2c, State: Running, Role: LEADER
I20260812 11:09:36.012049 21377 consensus_queue.cc:237] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } }
I20260812 11:09:36.014860 20381 catalog_manager.cc:5495] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 779b475921a940629f223a43e4cd3d2c (127.17.44.193). New cstate: current_term: 1 leader_uuid: "779b475921a940629f223a43e4cd3d2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "779b475921a940629f223a43e4cd3d2c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 } health_report { overall_health: UNKNOWN } } }
I20260812 11:09:36.058166 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:36.053401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34844 (local address 127.17.44.193:44271)
0812 11:09:36.053724 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:36.053730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:36.053741 (+ 11us) server_negotiation.cc:407] Connection header received
0812 11:09:36.053817 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:36.053820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:36.053872 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:36.053974 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:09:36.054354 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:36.055003 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:36.055855 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:36.056239 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:36.056719 (+ 480us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:36.057125 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:09:36.057616 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:09:36.057714 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:36.057883 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 11:09:36.057973 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":153,"threads_started":1}
I20260812 11:09:36.068521 21410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:36.064766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:35985 (local address 127.17.44.193:47233)
0812 11:09:36.065088 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:36.065101 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:36.065190 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:36.065524 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:36.065526 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:36.065533 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 11:09:36.065633 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:36.065637 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:36.066507 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:36.066510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:36.067321 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:36.067337 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:36.067781 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:36.067784 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:36.067868 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:36.067912 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 11:09:36.068251 (+ 339us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 11:09:36.068258 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 11:09:36.068333 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 11:09:36.068395 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":138,"threads_started":1}
I20260812 11:09:36.068785 21383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:36.064858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:47233 (local address 127.17.44.194:35985)
0812 11:09:36.064987 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:36.064992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:36.065190 (+ 198us) server_negotiation.cc:407] Connection header received
0812 11:09:36.065363 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:36.065365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:36.065418 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:36.065506 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:09:36.065771 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:36.066353 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:36.067453 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:36.067664 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:36.067784 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:36.068059 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:09:36.068134 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:09:36.068201 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:36.068411 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 11:09:36.068644 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 11:09:36.074353 21118 raft_consensus.cc:1261] T b7db81543a724175be0367508d05f5b9 P 5845f893ab0a4e999b4c30e7746f62dc [term 1 FOLLOWER]: Refusing update from remote peer 779b475921a940629f223a43e4cd3d2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:09:36.074561 20868 raft_consensus.cc:1261] T b7db81543a724175be0367508d05f5b9 P d181e17ef5d547bca505e1c5e5bbf543 [term 1 FOLLOWER]: Refusing update from remote peer 779b475921a940629f223a43e4cd3d2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:09:36.074822 21399 consensus_queue.cc:1035] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5845f893ab0a4e999b4c30e7746f62dc" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 41799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:09:36.075050 21377 consensus_queue.cc:1035] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d181e17ef5d547bca505e1c5e5bbf543" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:09:36.106228 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:36.102921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:38587 (local address 127.17.44.193:44271)
0812 11:09:36.103072 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:36.103077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:36.103318 (+ 241us) server_negotiation.cc:407] Connection header received
0812 11:09:36.103451 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:36.103454 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:36.103510 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:36.103587 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:09:36.103830 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:36.104364 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:36.105205 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:36.105440 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:36.105548 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:36.105801 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:09:36.105877 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:09:36.105953 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:36.106050 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 11:09:36.106100 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:09:36.106288 21420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:36.102863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:44271 (local address 127.17.44.194:38587)
0812 11:09:36.103235 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:36.103251 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:36.103362 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:36.103597 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:36.103600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:36.103608 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 11:09:36.103718 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:36.103724 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:36.104472 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:36.104476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:36.105083 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:36.105092 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:36.105560 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:36.105563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:36.105650 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:36.105682 (+ 32us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 11:09:36.105958 (+ 276us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 11:09:36.105967 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 11:09:36.106031 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 11:09:36.106124 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":144,"threads_started":1}
W20260812 11:09:36.107012 21325 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 11:09:37.674081 20385 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:09:37.674240 20385 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:09:37.677342 20385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 06548a5ccf064f4bb3a3b9ca51faba19: Sending DeleteTablet for 3 replicas of tablet b7db81543a724175be0367508d05f5b9
I20260812 11:09:37.678128 20854 tablet_service.cc:1514] Processing DeleteTablet for tablet b7db81543a724175be0367508d05f5b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:09:37 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:52814
I20260812 11:09:37.678193 21077 tablet_service.cc:1514] Processing DeleteTablet for tablet b7db81543a724175be0367508d05f5b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:09:37 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:33390
I20260812 11:09:37.678234 20622 tablet_service.cc:1514] Processing DeleteTablet for tablet b7db81543a724175be0367508d05f5b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:09:37 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:34834
I20260812 11:09:37.678555 21465 tablet_replica.cc:330] T b7db81543a724175be0367508d05f5b9 P d181e17ef5d547bca505e1c5e5bbf543: stopping tablet replica
I20260812 11:09:37.678689 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 20506
I20260812 11:09:37.678709 21465 raft_consensus.cc:2229] T b7db81543a724175be0367508d05f5b9 P d181e17ef5d547bca505e1c5e5bbf543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:09:37.678752 21468 tablet_replica.cc:330] T b7db81543a724175be0367508d05f5b9 P 779b475921a940629f223a43e4cd3d2c: stopping tablet replica
I20260812 11:09:37.683857 21465 raft_consensus.cc:2258] T b7db81543a724175be0367508d05f5b9 P d181e17ef5d547bca505e1c5e5bbf543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:09:37.684013 21467 tablet_replica.cc:330] T b7db81543a724175be0367508d05f5b9 P 5845f893ab0a4e999b4c30e7746f62dc: stopping tablet replica
I20260812 11:09:37.684168 21467 raft_consensus.cc:2229] T b7db81543a724175be0367508d05f5b9 P 5845f893ab0a4e999b4c30e7746f62dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:09:37.684296 21467 raft_consensus.cc:2258] T b7db81543a724175be0367508d05f5b9 P 5845f893ab0a4e999b4c30e7746f62dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:09:37.685233 21467 ts_tablet_manager.cc:1905] T b7db81543a724175be0367508d05f5b9 P 5845f893ab0a4e999b4c30e7746f62dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:09:37.685483 21465 ts_tablet_manager.cc:1905] T b7db81543a724175be0367508d05f5b9 P d181e17ef5d547bca505e1c5e5bbf543: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:09:37.688264 21467 ts_tablet_manager.cc:1918] T b7db81543a724175be0367508d05f5b9 P 5845f893ab0a4e999b4c30e7746f62dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 11:09:37.688370 21467 log.cc:1196] T b7db81543a724175be0367508d05f5b9 P 5845f893ab0a4e999b4c30e7746f62dc: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/b7db81543a724175be0367508d05f5b9
I20260812 11:09:37.688498 21465 ts_tablet_manager.cc:1918] T b7db81543a724175be0367508d05f5b9 P d181e17ef5d547bca505e1c5e5bbf543: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 11:09:37.688591 21465 log.cc:1196] T b7db81543a724175be0367508d05f5b9 P d181e17ef5d547bca505e1c5e5bbf543: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/b7db81543a724175be0367508d05f5b9
I20260812 11:09:37.688702 21467 ts_tablet_manager.cc:1939] T b7db81543a724175be0367508d05f5b9 P 5845f893ab0a4e999b4c30e7746f62dc: Deleting consensus metadata
I20260812 11:09:37.688925 21465 ts_tablet_manager.cc:1939] T b7db81543a724175be0367508d05f5b9 P d181e17ef5d547bca505e1c5e5bbf543: Deleting consensus metadata
I20260812 11:09:37.689592 20356 catalog_manager.cc:4841] TS 5845f893ab0a4e999b4c30e7746f62dc (127.17.44.195:41799): tablet b7db81543a724175be0367508d05f5b9 (table test-table [id=9c41f16f18d547ee888a0d1339626212]) successfully deleted
I20260812 11:09:37.689688 20360 catalog_manager.cc:4841] TS d181e17ef5d547bca505e1c5e5bbf543 (127.17.44.194:35985): tablet b7db81543a724175be0367508d05f5b9 (table test-table [id=9c41f16f18d547ee888a0d1339626212]) successfully deleted
I20260812 11:09:37.694276 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 20738
W20260812 11:09:37.694259 20357 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.44.193:44271 (error 108)
I20260812 11:09:37.704609 21471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:37.704046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:44271 (local address 127.0.0.1:34848)
0812 11:09:37.704362 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:37.704463 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.193:44271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":113,"threads_started":1}
W20260812 11:09:37.704806 20357 catalog_manager.cc:4584] TS 779b475921a940629f223a43e4cd3d2c (127.17.44.193:44271): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b7db81543a724175be0367508d05f5b9: Network error: Client connection negotiation failed: client connection to 127.17.44.193:44271: connect: Connection refused (error 111)
I20260812 11:09:37.711336 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 20977
I20260812 11:09:37.727097 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 20320
2026-08-12T11:09:37Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6877 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:09:37 dist-test-slave-drl0 krb5kdc[21474](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:09:37 dist-test-slave-drl0 krb5kdc[21474](info): set up 2 sockets
Aug 12 11:09:37 dist-test-slave-drl0 krb5kdc[21474](info): commencing operation
krb5kdc: starting...
W20260812 11:09:39.057533 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:09:39 dist-test-slave-drl0 krb5kdc[21474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:09:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:09:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:09:39.202410 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:33037
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:33037
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:39.330005 21502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:39.330453 21502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:39.330559 21502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:39.336820 21502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:09:39.337040 21502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:39.337123 21502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:39.337149 21502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:09:39.337168 21502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:09:39.343788 21502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:33037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:33037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:39.344959 21502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:39.346048 21502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:39.357471 21510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:39.357441 21512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:39.357417 21514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:39.357506 21502 server_base.cc:1034] running on GCE node
I20260812 11:09:39.358726 21502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:39.360208 21502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:39.361444 21502 hybrid_clock.cc:648] HybridClock initialized: now 1786532979361410 us; error 42 us; skew 500 ppm
Aug 12 11:09:39 dist-test-slave-drl0 krb5kdc[21474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532979, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:39.365319 21502 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:09:39.366285 21502 webserver.cc:460] Webserver started at http://127.17.44.254:45473/ using document root <none> and password file <none>
I20260812 11:09:39.366894 21502 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:39.367041 21502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:39.367291 21502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:39.369421 21502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "0d60dd412cbb4c0b9e875e2916a28fd8"
format_stamp: "Formatted at 2026-08-12 11:09:39 on dist-test-slave-drl0"
server_key: "55374cd8336935adca46b5e43a8b47f7"
server_key_iv: "591ab8a43667e11289f2b67d0c6d074c"
server_key_version: "encryptionkey@0"
I20260812 11:09:39.370081 21502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "0d60dd412cbb4c0b9e875e2916a28fd8"
format_stamp: "Formatted at 2026-08-12 11:09:39 on dist-test-slave-drl0"
server_key: "55374cd8336935adca46b5e43a8b47f7"
server_key_iv: "591ab8a43667e11289f2b67d0c6d074c"
server_key_version: "encryptionkey@0"
I20260812 11:09:39.374239 21502 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 11:09:39.377044 21525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:39.378515 21502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:09:39.378654 21502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "0d60dd412cbb4c0b9e875e2916a28fd8"
format_stamp: "Formatted at 2026-08-12 11:09:39 on dist-test-slave-drl0"
server_key: "55374cd8336935adca46b5e43a8b47f7"
server_key_iv: "591ab8a43667e11289f2b67d0c6d074c"
server_key_version: "encryptionkey@0"
I20260812 11:09:39.378763 21502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:39.404392 21502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:39.408869 21502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:39.409169 21502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:39.417222 21502 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:33037
I20260812 11:09:39.417294 21611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:33037 every 8 connection(s)
I20260812 11:09:39.418305 21502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:09:39.420953 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 21502
I20260812 11:09:39.421094 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:09:39.421337 21612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:39.421370 17587 external_mini_cluster.cc:1428] Setting key 7f1d66f219431f87e06c9fce10a16ddd
I20260812 11:09:39.428881 21612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d60dd412cbb4c0b9e875e2916a28fd8: Bootstrap starting.
I20260812 11:09:39.432653 21612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d60dd412cbb4c0b9e875e2916a28fd8: Neither blocks nor log segments found. Creating new log.
Aug 12 11:09:39 dist-test-slave-drl0 krb5kdc[21474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:39.433761 21612 log.cc:824] T 00000000000000000000000000000000 P 0d60dd412cbb4c0b9e875e2916a28fd8: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:39.436600 21612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d60dd412cbb4c0b9e875e2916a28fd8: No bootstrap required, opened a new log
I20260812 11:09:39.440253 21616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:39.423090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39646 (local address 127.17.44.254:33037)
0812 11:09:39.423682 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:39.423704 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:39.423755 (+ 51us) server_negotiation.cc:407] Connection header received
0812 11:09:39.424637 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:39.424657 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:39.424987 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:39.425270 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:39.426262 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:39.427532 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:39.429186 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:39.429667 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:39.429811 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:39.433584 (+ 3773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:39.433593 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:39.433615 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:39.433667 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:39.436276 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:39.436930 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:39.436937 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:39.436945 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:39.437232 (+ 287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:39.437611 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:39.437615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:39.437617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:39.438159 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:39.438340 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:39.438686 (+ 346us) server_negotiation.cc:299] Negotiation successful
0812 11:09:39.439067 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":177,"threads_started":1}
I20260812 11:09:39.441795 21612 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d60dd412cbb4c0b9e875e2916a28fd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d60dd412cbb4c0b9e875e2916a28fd8" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33037 } }
I20260812 11:09:39.442021 21612 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d60dd412cbb4c0b9e875e2916a28fd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:39.442067 21612 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d60dd412cbb4c0b9e875e2916a28fd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d60dd412cbb4c0b9e875e2916a28fd8, State: Initialized, Role: FOLLOWER
I20260812 11:09:39.442553 21612 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d60dd412cbb4c0b9e875e2916a28fd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d60dd412cbb4c0b9e875e2916a28fd8" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33037 } }
I20260812 11:09:39.442683 21612 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d60dd412cbb4c0b9e875e2916a28fd8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:09:39.442749 21612 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d60dd412cbb4c0b9e875e2916a28fd8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:09:39.442857 21612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d60dd412cbb4c0b9e875e2916a28fd8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:39.443994 21612 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d60dd412cbb4c0b9e875e2916a28fd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d60dd412cbb4c0b9e875e2916a28fd8" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33037 } }
I20260812 11:09:39.444293 21612 leader_election.cc:302] T 00000000000000000000000000000000 P 0d60dd412cbb4c0b9e875e2916a28fd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d60dd412cbb4c0b9e875e2916a28fd8; no voters:
I20260812 11:09:39.444577 21612 leader_election.cc:288] T 00000000000000000000000000000000 P 0d60dd412cbb4c0b9e875e2916a28fd8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:09:39.444908 21619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d60dd412cbb4c0b9e875e2916a28fd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:39.445319 21619 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d60dd412cbb4c0b9e875e2916a28fd8 [term 1 LEADER]: Becoming Leader. State: Replica: 0d60dd412cbb4c0b9e875e2916a28fd8, State: Running, Role: LEADER
I20260812 11:09:39.445768 21619 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d60dd412cbb4c0b9e875e2916a28fd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d60dd412cbb4c0b9e875e2916a28fd8" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33037 } }
I20260812 11:09:39.445832 21612 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d60dd412cbb4c0b9e875e2916a28fd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:09:39.447865 21623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d60dd412cbb4c0b9e875e2916a28fd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d60dd412cbb4c0b9e875e2916a28fd8. Latest consensus state: current_term: 1 leader_uuid: "0d60dd412cbb4c0b9e875e2916a28fd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d60dd412cbb4c0b9e875e2916a28fd8" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33037 } } }
I20260812 11:09:39.448014 21623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d60dd412cbb4c0b9e875e2916a28fd8 [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:39.448347 21621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d60dd412cbb4c0b9e875e2916a28fd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d60dd412cbb4c0b9e875e2916a28fd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d60dd412cbb4c0b9e875e2916a28fd8" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33037 } } }
I20260812 11:09:39.449065 21621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d60dd412cbb4c0b9e875e2916a28fd8 [sys.catalog]: This master's current role is: LEADER
W20260812 11:09:39.451835 21642 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0d60dd412cbb4c0b9e875e2916a28fd8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:09:39.451946 21642 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:09:39.452032 21644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:09:39.453042 21644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:09:39.460253 21644 catalog_manager.cc:1347] Generated new cluster ID: 884c3efc6d034fed8f05371bc708da31
I20260812 11:09:39.460336 21644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:09:39.480592 21644 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:09:39.481673 21644 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:09:39.490077 21644 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d60dd412cbb4c0b9e875e2916a28fd8: Generated new TSK 0
I20260812 11:09:39.490883 21644 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
I20260812 11:09:39.531157 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:33037
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:39.661442 21650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:39.661693 21650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:39.661737 21650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:39.665930 21650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:39.665997 21650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:39.666057 21650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:09:39.671684 21650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:33037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:39.672722 21650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:39.673765 21650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:39.683636 21658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:39.683580 21656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:39.683601 21662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:39.684284 21650 server_base.cc:1034] running on GCE node
I20260812 11:09:39.684720 21650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:39.685392 21650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:39.686614 21650 hybrid_clock.cc:648] HybridClock initialized: now 1786532979686584 us; error 43 us; skew 500 ppm
Aug 12 11:09:39 dist-test-slave-drl0 krb5kdc[21474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532979, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:39.689662 21650 init.cc:377] Logged in from keytab as kudu/127.17.44.193@KRBTEST.COM (short username kudu)
I20260812 11:09:39.690510 21650 webserver.cc:460] Webserver started at http://127.17.44.193:33509/ using document root <none> and password file <none>
I20260812 11:09:39.691138 21650 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:39.691226 21650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:39.691454 21650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:39.693472 21650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "241964d18db5465a9ede37a43c3bf7ea"
format_stamp: "Formatted at 2026-08-12 11:09:39 on dist-test-slave-drl0"
server_key: "dd3552cd6b82de4957e7190eb1af170f"
server_key_iv: "5cfa8b2f139da4cfca3bec4d4e27427b"
server_key_version: "encryptionkey@0"
I20260812 11:09:39.694023 21650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "241964d18db5465a9ede37a43c3bf7ea"
format_stamp: "Formatted at 2026-08-12 11:09:39 on dist-test-slave-drl0"
server_key: "dd3552cd6b82de4957e7190eb1af170f"
server_key_iv: "5cfa8b2f139da4cfca3bec4d4e27427b"
server_key_version: "encryptionkey@0"
I20260812 11:09:39.698342 21650 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 11:09:39.701601 21669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:39.702914 21650 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:09:39.703133 21650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "241964d18db5465a9ede37a43c3bf7ea"
format_stamp: "Formatted at 2026-08-12 11:09:39 on dist-test-slave-drl0"
server_key: "dd3552cd6b82de4957e7190eb1af170f"
server_key_iv: "5cfa8b2f139da4cfca3bec4d4e27427b"
server_key_version: "encryptionkey@0"
I20260812 11:09:39.703245 21650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:39.722537 21650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:39.727571 21650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:39.727795 21650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:39.728433 21650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:39.729310 21650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:39.729382 21650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:39.729455 21650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:39.729496 21650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:39.739439 21650 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:34417
I20260812 11:09:39.739467 21854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:34417 every 8 connection(s)
I20260812 11:09:39.740368 21650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:09:39.749192 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 21650
I20260812 11:09:39.749333 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:09:39.749645 17587 external_mini_cluster.cc:1428] Setting key f71f78e741a8f4637dcd33249b853d25
Aug 12 11:09:39 dist-test-slave-drl0 krb5kdc[21474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532979, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:39.756250 21616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:39.742501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:56777 (local address 127.17.44.254:33037)
0812 11:09:39.742656 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:39.742662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:39.743580 (+ 918us) server_negotiation.cc:407] Connection header received
0812 11:09:39.745043 (+ 1463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:39.745046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:39.745100 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:39.745190 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:39.746584 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:39.747229 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:39.748923 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:39.749188 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:39.749322 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:39.752006 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:39.752009 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:39.752011 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:39.752033 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:39.754088 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:39.754627 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:39.754631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:39.754633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:39.754713 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:39.755104 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:39.755109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:39.755110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:39.755332 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:39.755441 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:39.755947 (+ 506us) server_negotiation.cc:299] Negotiation successful
0812 11:09:39.756077 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 11:09:39.757052 21860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:39.742890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33037 (local address 127.17.44.193:56777)
0812 11:09:39.743433 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:39.743476 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:39.744747 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:39.745460 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:39.745469 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:39.745846 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:39.746359 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:39.746373 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:39.747484 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:39.747489 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:39.748778 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:39.748793 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:39.749394 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:39.749399 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:39.749602 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:39.750338 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:39.750355 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:39.751832 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:39.754228 (+ 2396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:39.754234 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:39.754255 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:39.754520 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:39.754809 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:39.754813 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:39.754815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:39.755009 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:39.755442 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:39.755451 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:39.755695 (+ 244us) client_negotiation.cc:769] Sending connection context
0812 11:09:39.755890 (+ 195us) client_negotiation.cc:240] Negotiation successful
0812 11:09:39.756214 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":108,"threads_started":1}
I20260812 11:09:39.758606 21858 heartbeater.cc:347] Connected to a master server at 127.17.44.254:33037
I20260812 11:09:39.758891 21858 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:39.759558 21858 heartbeater.cc:511] Master 127.17.44.254:33037 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.44.194@KRBTEST.COM; defaulting to no policy
I20260812 11:09:39.761404 21553 ts_manager.cc:194] Registered new tserver with Master: 241964d18db5465a9ede37a43c3bf7ea (127.17.44.193:34417)
I20260812 11:09:39.762555 21553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.193@KRBTEST.COM'} at 127.17.44.193:56777
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
I20260812 11:09:39.780071 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:33037
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:39.902724 21866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:39.903012 21866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:39.903061 21866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:39.907204 21866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:39.907406 21866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:39.907511 21866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:09:39.912999 21866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:33037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:39.914017 21866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:39.915040 21866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:39.925489 21873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:39.925540 21872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:39.925549 21866 server_base.cc:1034] running on GCE node
W20260812 11:09:39.925699 21876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:39.926090 21866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:39.926690 21866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:39.927863 21866 hybrid_clock.cc:648] HybridClock initialized: now 1786532979927853 us; error 34 us; skew 500 ppm
Aug 12 11:09:39 dist-test-slave-drl0 krb5kdc[21474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532979, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:39.931100 21866 init.cc:377] Logged in from keytab as kudu/127.17.44.194@KRBTEST.COM (short username kudu)
I20260812 11:09:39.932014 21866 webserver.cc:460] Webserver started at http://127.17.44.194:43061/ using document root <none> and password file <none>
I20260812 11:09:39.932601 21866 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:39.932693 21866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:39.933036 21866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:39.935268 21866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "64785d71cc4f433ab0fe4e8e2b6d3b58"
format_stamp: "Formatted at 2026-08-12 11:09:39 on dist-test-slave-drl0"
server_key: "545b383aade8e8b3a1a306ca5a9756c9"
server_key_iv: "0ecb57c75eb9a9adef143c05cdfb51e5"
server_key_version: "encryptionkey@0"
I20260812 11:09:39.935875 21866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "64785d71cc4f433ab0fe4e8e2b6d3b58"
format_stamp: "Formatted at 2026-08-12 11:09:39 on dist-test-slave-drl0"
server_key: "545b383aade8e8b3a1a306ca5a9756c9"
server_key_iv: "0ecb57c75eb9a9adef143c05cdfb51e5"
server_key_version: "encryptionkey@0"
I20260812 11:09:39.939533 21866 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:09:39.942138 21885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:39.943447 21866 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:09:39.943734 21866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "64785d71cc4f433ab0fe4e8e2b6d3b58"
format_stamp: "Formatted at 2026-08-12 11:09:39 on dist-test-slave-drl0"
server_key: "545b383aade8e8b3a1a306ca5a9756c9"
server_key_iv: "0ecb57c75eb9a9adef143c05cdfb51e5"
server_key_version: "encryptionkey@0"
I20260812 11:09:39.943864 21866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:39.958560 21866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:39.962720 21866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:39.963047 21866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:39.963702 21866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:39.964619 21866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:39.964694 21866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:39.964763 21866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:39.964818 21866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:39.975530 21866 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:46487
I20260812 11:09:39.975625 22060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:46487 every 8 connection(s)
I20260812 11:09:39.976573 21866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:09:39.978178 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 21866
I20260812 11:09:39.978310 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:09:39.978632 17587 external_mini_cluster.cc:1428] Setting key 7e71121087c2c2998b892ce070bd7ce3
Aug 12 11:09:39 dist-test-slave-drl0 krb5kdc[21474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532979, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.44.195@KRBTEST.COM; defaulting to no policy
I20260812 11:09:39.993841 21616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:39.978517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:33209 (local address 127.17.44.254:33037)
0812 11:09:39.978682 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:39.978687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:39.980025 (+ 1338us) server_negotiation.cc:407] Connection header received
0812 11:09:39.980969 (+ 944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:39.980975 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:39.981036 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:39.981132 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:39.982552 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:39.983133 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:39.984296 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:39.984688 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:39.984821 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:39.988175 (+ 3354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:39.988185 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:39.988192 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:39.988258 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:39.991408 (+ 3150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:39.992106 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:39.992112 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:39.992114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:39.992226 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:39.992637 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:39.992642 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:39.992643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:39.992924 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:39.993032 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:39.993591 (+ 559us) server_negotiation.cc:299] Negotiation successful
0812 11:09:39.993701 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 11:09:39.995051 22066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:39.979029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33037 (local address 127.17.44.194:33209)
0812 11:09:39.979856 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:39.979913 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:39.980760 (+ 847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:39.981310 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:39.981317 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:39.981814 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:39.982409 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:39.982424 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:39.983273 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:39.983277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:39.984166 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:39.984175 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:39.984890 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:39.984893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:39.985048 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:39.985738 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:39.985756 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:39.987559 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:39.991566 (+ 4007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:39.991627 (+ 61us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:39.991646 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:39.991976 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:39.992356 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:39.992359 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:39.992361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:39.992510 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:39.993063 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:39.993069 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:39.993329 (+ 260us) client_negotiation.cc:769] Sending connection context
0812 11:09:39.993498 (+ 169us) client_negotiation.cc:240] Negotiation successful
0812 11:09:39.993867 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":146,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.195@KRBTEST.COM" created.
I20260812 11:09:39.996320 22064 heartbeater.cc:347] Connected to a master server at 127.17.44.254:33037
I20260812 11:09:39.996580 22064 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:39.997174 22064 heartbeater.cc:511] Master 127.17.44.254:33037 requested a full tablet report, sending...
I20260812 11:09:39.998361 21553 ts_manager.cc:194] Registered new tserver with Master: 64785d71cc4f433ab0fe4e8e2b6d3b58 (127.17.44.194:46487)
I20260812 11:09:39.999042 21553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.194@KRBTEST.COM'} at 127.17.44.194:33209
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
I20260812 11:09:40.014495 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:33037
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:40.147557 22071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:40.147876 22071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:40.147969 22071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:40.152391 22071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:40.152477 22071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:40.152567 22071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:09:40.158013 22071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:33037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:40.159807 22071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:40.161460 22071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:40.173062 22088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:40.173086 22079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:40.173060 22084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:40.173648 22071 server_base.cc:1034] running on GCE node
I20260812 11:09:40.174307 22071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:40.175062 22071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:40.176263 22071 hybrid_clock.cc:648] HybridClock initialized: now 1786532980176246 us; error 27 us; skew 500 ppm
Aug 12 11:09:40 dist-test-slave-drl0 krb5kdc[21474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532980, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:40.179594 22071 init.cc:377] Logged in from keytab as kudu/127.17.44.195@KRBTEST.COM (short username kudu)
I20260812 11:09:40.180888 22071 webserver.cc:460] Webserver started at http://127.17.44.195:34339/ using document root <none> and password file <none>
I20260812 11:09:40.181506 22071 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:40.181571 22071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:40.182363 22071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:40.185233 22071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "5e8f1e50933c49f781badd02a0fb2c20"
format_stamp: "Formatted at 2026-08-12 11:09:40 on dist-test-slave-drl0"
server_key: "1562daf69a710350588a7a3e29f53889"
server_key_iv: "6a55c4b00db8ac2d0222943329b46b8f"
server_key_version: "encryptionkey@0"
I20260812 11:09:40.185931 22071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "5e8f1e50933c49f781badd02a0fb2c20"
format_stamp: "Formatted at 2026-08-12 11:09:40 on dist-test-slave-drl0"
server_key: "1562daf69a710350588a7a3e29f53889"
server_key_iv: "6a55c4b00db8ac2d0222943329b46b8f"
server_key_version: "encryptionkey@0"
I20260812 11:09:40.190621 22071 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260812 11:09:40.193562 22098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:40.195065 22071 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:09:40.195175 22071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "5e8f1e50933c49f781badd02a0fb2c20"
format_stamp: "Formatted at 2026-08-12 11:09:40 on dist-test-slave-drl0"
server_key: "1562daf69a710350588a7a3e29f53889"
server_key_iv: "6a55c4b00db8ac2d0222943329b46b8f"
server_key_version: "encryptionkey@0"
I20260812 11:09:40.195318 22071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:40.216127 22071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:40.219970 22071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:40.220171 22071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:40.220804 22071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:40.221676 22071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:40.221724 22071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:40.221786 22071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:40.221906 22071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:40.232338 22071 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:39749
I20260812 11:09:40.232354 22279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:39749 every 8 connection(s)
I20260812 11:09:40.233254 22071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
Aug 12 11:09:40 dist-test-slave-drl0 krb5kdc[21474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532980, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:40.243093 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 22071
I20260812 11:09:40.243222 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:09:40.243547 17587 external_mini_cluster.cc:1428] Setting key 3f48f0dcb05b297a72a0501403df12a3
I20260812 11:09:40.249662 21616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:40.235346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:51045 (local address 127.17.44.254:33037)
0812 11:09:40.235542 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:40.235546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:40.236494 (+ 948us) server_negotiation.cc:407] Connection header received
0812 11:09:40.237366 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:40.237369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:40.237417 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:40.237496 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:40.238621 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:40.239429 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:40.240583 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:40.240819 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:40.240928 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:40.243962 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:40.243971 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:40.243973 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:40.243997 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:40.246445 (+ 2448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:40.247267 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:40.247272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:40.247275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:40.247389 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:40.247982 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:40.247985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:40.247987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:40.248275 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:40.248399 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:40.249321 (+ 922us) server_negotiation.cc:299] Negotiation successful
0812 11:09:40.249531 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260812 11:09:40.250861 22282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:40.235648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33037 (local address 127.17.44.195:51045)
0812 11:09:40.236337 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:40.236376 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:40.237163 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:40.237649 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:40.237656 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:40.237991 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:40.238463 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:40.238476 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:40.239576 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:40.239580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:40.240456 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:40.240466 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:40.240983 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:40.240987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:40.241154 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:40.241794 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:40.241811 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:40.243782 (+ 1971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:40.246623 (+ 2841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:40.246631 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:40.246652 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:40.247141 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:40.247525 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:40.247530 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:40.247533 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:40.247815 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:40.248521 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:40.248532 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:40.248940 (+ 408us) client_negotiation.cc:769] Sending connection context
0812 11:09:40.249206 (+ 266us) client_negotiation.cc:240] Negotiation successful
0812 11:09:40.249695 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":122,"threads_started":1}
I20260812 11:09:40.252635 22280 heartbeater.cc:347] Connected to a master server at 127.17.44.254:33037
I20260812 11:09:40.252938 22280 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:40.253504 22280 heartbeater.cc:511] Master 127.17.44.254:33037 requested a full tablet report, sending...
I20260812 11:09:40.254781 21553 ts_manager.cc:194] Registered new tserver with Master: 5e8f1e50933c49f781badd02a0fb2c20 (127.17.44.195:39749)
I20260812 11:09:40.255445 21553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.195@KRBTEST.COM'} at 127.17.44.195:51045
I20260812 11:09:40.259814 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 11:09:40.278000 21616 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:09:40.276478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39650 (local address 127.17.44.254:33037)
0812 11:09:40.276640 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:40.276645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:40.276805 (+ 160us) server_negotiation.cc:407] Connection header received
0812 11:09:40.276981 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:40.276984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:40.277028 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:40.277110 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:40.277891 (+ 781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39650: BlockingRecv error: recv got EOF from 127.0.0.1:39650 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 11:09:40.279139 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 21650
I20260812 11:09:40.302848 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 21866
I20260812 11:09:40.321826 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 22071
I20260812 11:09:40.336650 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 21502
2026-08-12T11:09:40Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2604 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:09:40 dist-test-slave-drl0 krb5kdc[22296](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:09:40 dist-test-slave-drl0 krb5kdc[22296](info): set up 2 sockets
Aug 12 11:09:40 dist-test-slave-drl0 krb5kdc[22296](info): commencing operation
krb5kdc: starting...
W20260812 11:09:41.629618 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:09:41 dist-test-slave-drl0 krb5kdc[22296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:09:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:09:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:09:41.772557 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:42621
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:42621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:41.893214 22323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:41.893599 22323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:41.893695 22323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:41.897820 22323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:09:41.897887 22323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:41.897908 22323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:41.897925 22323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:09:41.897941 22323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:09:41.903527 22323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:42621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:42621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:41.905138 22323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:41.906196 22323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:41.915138 22332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:41.915174 22334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:41.915241 22331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:41.915459 22323 server_base.cc:1034] running on GCE node
I20260812 11:09:41.916123 22323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:41.918108 22323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:41.919389 22323 hybrid_clock.cc:648] HybridClock initialized: now 1786532981919356 us; error 51 us; skew 500 ppm
Aug 12 11:09:41 dist-test-slave-drl0 krb5kdc[22296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532981, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:41.922286 22323 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:09:41.923237 22323 webserver.cc:460] Webserver started at http://127.17.44.254:40845/ using document root <none> and password file <none>
I20260812 11:09:41.923774 22323 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:41.923842 22323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:41.924010 22323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:41.926066 22323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "be7c3012326f4a9180269a563bb53b98"
format_stamp: "Formatted at 2026-08-12 11:09:41 on dist-test-slave-drl0"
server_key: "3a8c6cac4f9c5d5779392f48d9bf7919"
server_key_iv: "20aa704aa1819a9a59d4ae07bc56cec4"
server_key_version: "encryptionkey@0"
I20260812 11:09:41.926599 22323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "be7c3012326f4a9180269a563bb53b98"
format_stamp: "Formatted at 2026-08-12 11:09:41 on dist-test-slave-drl0"
server_key: "3a8c6cac4f9c5d5779392f48d9bf7919"
server_key_iv: "20aa704aa1819a9a59d4ae07bc56cec4"
server_key_version: "encryptionkey@0"
I20260812 11:09:41.931492 22323 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 11:09:41.934131 22345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:41.935710 22323 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:09:41.935818 22323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "be7c3012326f4a9180269a563bb53b98"
format_stamp: "Formatted at 2026-08-12 11:09:41 on dist-test-slave-drl0"
server_key: "3a8c6cac4f9c5d5779392f48d9bf7919"
server_key_iv: "20aa704aa1819a9a59d4ae07bc56cec4"
server_key_version: "encryptionkey@0"
I20260812 11:09:41.936025 22323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:41.950851 22323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:41.954474 22323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:41.954643 22323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:41.964359 22323 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:42621
I20260812 11:09:41.964700 22425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:42621 every 8 connection(s)
I20260812 11:09:41.965413 22323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:09:41.968765 22426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:41.970603 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 22323
I20260812 11:09:41.970738 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:09:41.971081 17587 external_mini_cluster.cc:1428] Setting key 10a6468665b6777d53130562f3955333
I20260812 11:09:41.976693 22426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be7c3012326f4a9180269a563bb53b98: Bootstrap starting.
I20260812 11:09:41.979466 22426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be7c3012326f4a9180269a563bb53b98: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:41.980522 22426 log.cc:824] T 00000000000000000000000000000000 P be7c3012326f4a9180269a563bb53b98: Log is configured to *not* fsync() on all Append() calls
Aug 12 11:09:41 dist-test-slave-drl0 krb5kdc[22296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:41.983445 22426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be7c3012326f4a9180269a563bb53b98: No bootstrap required, opened a new log
I20260812 11:09:41.986712 22429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:41.972949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49710 (local address 127.17.44.254:42621)
0812 11:09:41.973426 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:41.973457 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:41.973492 (+ 35us) server_negotiation.cc:407] Connection header received
0812 11:09:41.974129 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:41.974145 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:41.974459 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:41.974733 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:41.976075 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:41.977115 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:41.978314 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:41.978784 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:41.978922 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:41.982016 (+ 3094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:41.982022 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:41.982036 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:41.982071 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:41.983868 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:41.984316 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:41.984320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:41.984325 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:41.984433 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:41.984715 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:41.984729 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:41.984730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:41.985116 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:41.985341 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:41.985581 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 11:09:41.985872 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":170,"threads_started":1}
I20260812 11:09:41.987249 22426 raft_consensus.cc:348] T 00000000000000000000000000000000 P be7c3012326f4a9180269a563bb53b98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be7c3012326f4a9180269a563bb53b98" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42621 } }
I20260812 11:09:41.987507 22426 raft_consensus.cc:374] T 00000000000000000000000000000000 P be7c3012326f4a9180269a563bb53b98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:41.987583 22426 raft_consensus.cc:729] T 00000000000000000000000000000000 P be7c3012326f4a9180269a563bb53b98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be7c3012326f4a9180269a563bb53b98, State: Initialized, Role: FOLLOWER
I20260812 11:09:41.988085 22426 consensus_queue.cc:260] T 00000000000000000000000000000000 P be7c3012326f4a9180269a563bb53b98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be7c3012326f4a9180269a563bb53b98" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42621 } }
I20260812 11:09:41.988245 22426 raft_consensus.cc:388] T 00000000000000000000000000000000 P be7c3012326f4a9180269a563bb53b98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:09:41.988322 22426 raft_consensus.cc:482] T 00000000000000000000000000000000 P be7c3012326f4a9180269a563bb53b98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:09:41.988418 22426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be7c3012326f4a9180269a563bb53b98 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:41.989630 22426 raft_consensus.cc:504] T 00000000000000000000000000000000 P be7c3012326f4a9180269a563bb53b98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be7c3012326f4a9180269a563bb53b98" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42621 } }
I20260812 11:09:41.989997 22426 leader_election.cc:302] T 00000000000000000000000000000000 P be7c3012326f4a9180269a563bb53b98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be7c3012326f4a9180269a563bb53b98; no voters:
I20260812 11:09:41.990072 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:09:41.990372 22426 leader_election.cc:288] T 00000000000000000000000000000000 P be7c3012326f4a9180269a563bb53b98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:09:41.990506 22433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be7c3012326f4a9180269a563bb53b98 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:41.990765 22433 raft_consensus.cc:686] T 00000000000000000000000000000000 P be7c3012326f4a9180269a563bb53b98 [term 1 LEADER]: Becoming Leader. State: Replica: be7c3012326f4a9180269a563bb53b98, State: Running, Role: LEADER
I20260812 11:09:41.991103 22433 consensus_queue.cc:237] T 00000000000000000000000000000000 P be7c3012326f4a9180269a563bb53b98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be7c3012326f4a9180269a563bb53b98" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42621 } }
I20260812 11:09:41.991789 22426 sys_catalog.cc:564] T 00000000000000000000000000000000 P be7c3012326f4a9180269a563bb53b98 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:09:41.992295 22433 sys_catalog.cc:455] T 00000000000000000000000000000000 P be7c3012326f4a9180269a563bb53b98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be7c3012326f4a9180269a563bb53b98. Latest consensus state: current_term: 1 leader_uuid: "be7c3012326f4a9180269a563bb53b98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be7c3012326f4a9180269a563bb53b98" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42621 } } }
I20260812 11:09:41.992434 22433 sys_catalog.cc:458] T 00000000000000000000000000000000 P be7c3012326f4a9180269a563bb53b98 [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:41.992940 22436 sys_catalog.cc:455] T 00000000000000000000000000000000 P be7c3012326f4a9180269a563bb53b98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be7c3012326f4a9180269a563bb53b98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be7c3012326f4a9180269a563bb53b98" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42621 } } }
I20260812 11:09:41.993103 22436 sys_catalog.cc:458] T 00000000000000000000000000000000 P be7c3012326f4a9180269a563bb53b98 [sys.catalog]: This master's current role is: LEADER
W20260812 11:09:41.997018 22456 catalog_manager.cc:1558] T 00000000000000000000000000000000 P be7c3012326f4a9180269a563bb53b98: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:09:41.997114 22456 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:09:41.997231 22440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:09:41.998106 22440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:09:42.005154 22440 catalog_manager.cc:1347] Generated new cluster ID: dfe3242bae3b4424a6ae16b1180d12d5
I20260812 11:09:42.005301 22440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:09:42.012720 22429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:42.006987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49720 (local address 127.17.44.254:42621)
0812 11:09:42.007114 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:42.007119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:42.007339 (+ 220us) server_negotiation.cc:407] Connection header received
0812 11:09:42.007436 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:42.007444 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:42.007497 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:42.007598 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:42.008165 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:42.009168 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:42.010297 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:42.010668 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:42.010781 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:42.011719 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:42.011723 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:42.011726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:09:42.012276 (+ 550us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:42.012310 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:09:42.012330 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:42.012433 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:42.012501 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 11:09:42.012562 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 11:09:42.030601 22440 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:09:42.032322 22440 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:09:42.046607 22440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be7c3012326f4a9180269a563bb53b98: Generated new TSK 0
I20260812 11:09:42.047490 22440 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:09:42.056196 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 22323
2026-08-12T11:09:42Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1717 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:09:42 dist-test-slave-drl0 krb5kdc[22485](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:09:42 dist-test-slave-drl0 krb5kdc[22485](info): set up 2 sockets
Aug 12 11:09:42 dist-test-slave-drl0 krb5kdc[22485](info): commencing operation
krb5kdc: starting...
W20260812 11:09:43.346697 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.263s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:09:43 dist-test-slave-drl0 krb5kdc[22485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:09:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:09:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:09:43.479245 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:39869
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:43067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:39869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:43.602864 22507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:43.603199 22507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:43.603274 22507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:43.607467 22507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:09:43.607548 22507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:43.607584 22507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:43.607718 22507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:09:43.607815 22507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:09:43.615990 22507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:43067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:39869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:39869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:43.617427 22507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:43.618628 22507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:43.627684 22514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:43.627656 22513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:43.627696 22517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:43.628005 22507 server_base.cc:1034] running on GCE node
I20260812 11:09:43.628546 22507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:43.629657 22507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:43.631055 22507 hybrid_clock.cc:648] HybridClock initialized: now 1786532983631023 us; error 73 us; skew 500 ppm
Aug 12 11:09:43 dist-test-slave-drl0 krb5kdc[22485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532983, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:43.637758 22507 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:09:43.638887 22507 webserver.cc:460] Webserver started at http://127.17.44.254:37379/ using document root <none> and password file <none>
I20260812 11:09:43.639521 22507 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:43.639588 22507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:43.639762 22507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:43.641960 22507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "a2c29f76d10b4883a820aadbacf06e6b"
format_stamp: "Formatted at 2026-08-12 11:09:43 on dist-test-slave-drl0"
server_key: "9298488a5b21d4a772a59bc4d5f924f8"
server_key_iv: "d66038f35af463a4a732322bbb2b0680"
server_key_version: "encryptionkey@0"
I20260812 11:09:43.642495 22507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "a2c29f76d10b4883a820aadbacf06e6b"
format_stamp: "Formatted at 2026-08-12 11:09:43 on dist-test-slave-drl0"
server_key: "9298488a5b21d4a772a59bc4d5f924f8"
server_key_iv: "d66038f35af463a4a732322bbb2b0680"
server_key_version: "encryptionkey@0"
I20260812 11:09:43.646761 22507 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 11:09:43.650254 22539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:43.651821 22507 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:09:43.651978 22507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "a2c29f76d10b4883a820aadbacf06e6b"
format_stamp: "Formatted at 2026-08-12 11:09:43 on dist-test-slave-drl0"
server_key: "9298488a5b21d4a772a59bc4d5f924f8"
server_key_iv: "d66038f35af463a4a732322bbb2b0680"
server_key_version: "encryptionkey@0"
I20260812 11:09:43.652097 22507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:43.671195 22507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:43.675720 22507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:43.676035 22507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:43.684816 22507 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:39869
I20260812 11:09:43.685194 22631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:39869 every 8 connection(s)
I20260812 11:09:43.685864 22507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:09:43.686488 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 22507
I20260812 11:09:43.686640 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:09:43.687235 17587 external_mini_cluster.cc:1428] Setting key b8b262a0710bfe8d588fb1eeffd30ed2
I20260812 11:09:43.689677 22632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:43.697237 22632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2c29f76d10b4883a820aadbacf06e6b: Bootstrap starting.
Aug 12 11:09:43 dist-test-slave-drl0 krb5kdc[22485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:43.699607 22632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2c29f76d10b4883a820aadbacf06e6b: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:43.700400 22632 log.cc:824] T 00000000000000000000000000000000 P a2c29f76d10b4883a820aadbacf06e6b: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:43.702564 22632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2c29f76d10b4883a820aadbacf06e6b: No bootstrap required, opened a new log
I20260812 11:09:43.703419 22636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:43.689293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36312 (local address 127.17.44.254:39869)
0812 11:09:43.689718 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:43.689735 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:43.689771 (+ 36us) server_negotiation.cc:407] Connection header received
0812 11:09:43.690470 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:43.690486 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:43.690811 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:43.691157 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:43.692216 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:43.693219 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:43.694391 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:43.695044 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:43.695178 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:43.698369 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:43.698374 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:43.698388 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:43.698420 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:43.700443 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:43.701023 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:43.701029 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:43.701033 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:43.701145 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:43.701437 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:43.701441 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:43.701442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:43.701837 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:43.702031 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:43.702332 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 11:09:43.702609 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":139,"threads_started":1}
I20260812 11:09:43.707104 22632 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2c29f76d10b4883a820aadbacf06e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c29f76d10b4883a820aadbacf06e6b" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 39869 } }
I20260812 11:09:43.707320 22632 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2c29f76d10b4883a820aadbacf06e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:43.707381 22632 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2c29f76d10b4883a820aadbacf06e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2c29f76d10b4883a820aadbacf06e6b, State: Initialized, Role: FOLLOWER
I20260812 11:09:43.707801 22632 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2c29f76d10b4883a820aadbacf06e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c29f76d10b4883a820aadbacf06e6b" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 39869 } }
I20260812 11:09:43.707955 22632 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2c29f76d10b4883a820aadbacf06e6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:09:43.708039 22632 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2c29f76d10b4883a820aadbacf06e6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:09:43.708148 22632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2c29f76d10b4883a820aadbacf06e6b [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:43.709030 22632 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2c29f76d10b4883a820aadbacf06e6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c29f76d10b4883a820aadbacf06e6b" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 39869 } }
I20260812 11:09:43.709399 22632 leader_election.cc:302] T 00000000000000000000000000000000 P a2c29f76d10b4883a820aadbacf06e6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2c29f76d10b4883a820aadbacf06e6b; no voters:
I20260812 11:09:43.709807 22632 leader_election.cc:288] T 00000000000000000000000000000000 P a2c29f76d10b4883a820aadbacf06e6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:09:43.710223 22640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2c29f76d10b4883a820aadbacf06e6b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:43.710506 22640 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2c29f76d10b4883a820aadbacf06e6b [term 1 LEADER]: Becoming Leader. State: Replica: a2c29f76d10b4883a820aadbacf06e6b, State: Running, Role: LEADER
I20260812 11:09:43.710877 22640 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2c29f76d10b4883a820aadbacf06e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c29f76d10b4883a820aadbacf06e6b" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 39869 } }
I20260812 11:09:43.711366 22632 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2c29f76d10b4883a820aadbacf06e6b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:09:43.712954 22642 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2c29f76d10b4883a820aadbacf06e6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2c29f76d10b4883a820aadbacf06e6b. Latest consensus state: current_term: 1 leader_uuid: "a2c29f76d10b4883a820aadbacf06e6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c29f76d10b4883a820aadbacf06e6b" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 39869 } } }
I20260812 11:09:43.712954 22641 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2c29f76d10b4883a820aadbacf06e6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2c29f76d10b4883a820aadbacf06e6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c29f76d10b4883a820aadbacf06e6b" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 39869 } } }
I20260812 11:09:43.713116 22641 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2c29f76d10b4883a820aadbacf06e6b [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:43.713116 22642 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2c29f76d10b4883a820aadbacf06e6b [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:43.714233 22651 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:09:43.718070 22651 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:09:43.723917 22651 catalog_manager.cc:1347] Generated new cluster ID: 26d5ed7622a3408e9f2afc7d863ce49e
I20260812 11:09:43.723978 22651 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:09:43.733489 22651 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:09:43.734759 22651 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:09:43.754989 22651 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2c29f76d10b4883a820aadbacf06e6b: Generated new TSK 0
I20260812 11:09:43.755854 22651 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
I20260812 11:09:43.798218 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:39869
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:43067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:43.938489 22668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:43.938758 22668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:43.938802 22668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:43.942772 22668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:43.942837 22668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:43.942895 22668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:09:43.949312 22668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:43067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:39869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:43.950409 22668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:43.951524 22668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:43.963840 22678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:43.963825 22683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:43.963825 22675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:43.964249 22668 server_base.cc:1034] running on GCE node
I20260812 11:09:43.964702 22668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:43.965372 22668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:43.966588 22668 hybrid_clock.cc:648] HybridClock initialized: now 1786532983966537 us; error 64 us; skew 500 ppm
Aug 12 11:09:43 dist-test-slave-drl0 krb5kdc[22485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532983, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:43.969657 22668 init.cc:377] Logged in from keytab as kudu/127.17.44.193@KRBTEST.COM (short username kudu)
I20260812 11:09:43.970484 22668 webserver.cc:460] Webserver started at http://127.17.44.193:38475/ using document root <none> and password file <none>
I20260812 11:09:43.971062 22668 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:43.971131 22668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:43.971319 22668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:43.973260 22668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "55ecc1bff4564e92811a472021bf6bce"
format_stamp: "Formatted at 2026-08-12 11:09:43 on dist-test-slave-drl0"
server_key: "0e25ebf910bc9823fc9de1a5f7e43de1"
server_key_iv: "04406e486d99db057ac875789c1e7a12"
server_key_version: "encryptionkey@0"
I20260812 11:09:43.973853 22668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "55ecc1bff4564e92811a472021bf6bce"
format_stamp: "Formatted at 2026-08-12 11:09:43 on dist-test-slave-drl0"
server_key: "0e25ebf910bc9823fc9de1a5f7e43de1"
server_key_iv: "04406e486d99db057ac875789c1e7a12"
server_key_version: "encryptionkey@0"
I20260812 11:09:43.978190 22668 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 11:09:43.981225 22696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:43.983100 22668 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:09:43.983234 22668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "55ecc1bff4564e92811a472021bf6bce"
format_stamp: "Formatted at 2026-08-12 11:09:43 on dist-test-slave-drl0"
server_key: "0e25ebf910bc9823fc9de1a5f7e43de1"
server_key_iv: "04406e486d99db057ac875789c1e7a12"
server_key_version: "encryptionkey@0"
I20260812 11:09:43.983325 22668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:44.024245 22668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:44.027709 22668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:44.027916 22668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:44.028674 22668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:44.029736 22668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:44.029794 22668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:44.029947 22668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:44.030014 22668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:44.042625 22668 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:43297
I20260812 11:09:44.042708 22882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:43297 every 8 connection(s)
I20260812 11:09:44.043658 22668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:09:44.047371 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 22668
I20260812 11:09:44.047698 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:09:44.048321 17587 external_mini_cluster.cc:1428] Setting key 240fc1d33a96b209d6b7cb8fddce17cb
Aug 12 11:09:44 dist-test-slave-drl0 krb5kdc[22485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532983, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.44.194@KRBTEST.COM; defaulting to no policy
I20260812 11:09:44.061252 22636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:44.045776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:37831 (local address 127.17.44.254:39869)
0812 11:09:44.045947 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:44.045951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:44.046980 (+ 1029us) server_negotiation.cc:407] Connection header received
0812 11:09:44.048849 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:44.048855 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:44.048946 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:44.049206 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:44.050592 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:44.051275 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:44.053225 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:44.053674 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:44.053845 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:44.056879 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:44.056887 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:44.056892 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:44.056947 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:44.058969 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:44.059506 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:44.059512 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:44.059517 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:44.059622 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:44.059991 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:44.059995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:44.059997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:44.060241 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:44.060362 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:44.061010 (+ 648us) server_negotiation.cc:299] Negotiation successful
0812 11:09:44.061125 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 11:09:44.062049 22886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:44.046126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:39869 (local address 127.17.44.193:37831)
0812 11:09:44.046814 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:44.046854 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:44.048498 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:44.049362 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:44.049371 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:44.049750 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:44.050403 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:44.050424 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:44.051433 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:44.051438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:44.053043 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:44.053061 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:44.053898 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:44.053902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:44.054064 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:44.054784 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:44.054802 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:44.056599 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:44.059092 (+ 2493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:44.059097 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:44.059115 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:44.059390 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:44.059741 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:44.059744 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:44.059745 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:44.059887 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:44.060415 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:44.060424 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:44.060749 (+ 325us) client_negotiation.cc:769] Sending connection context
0812 11:09:44.060929 (+ 180us) client_negotiation.cc:240] Negotiation successful
0812 11:09:44.061199 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":121,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.194@KRBTEST.COM" created.
I20260812 11:09:44.063611 22883 heartbeater.cc:347] Connected to a master server at 127.17.44.254:39869
I20260812 11:09:44.063879 22883 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:44.064386 22883 heartbeater.cc:511] Master 127.17.44.254:39869 requested a full tablet report, sending...
I20260812 11:09:44.066023 22573 ts_manager.cc:194] Registered new tserver with Master: 55ecc1bff4564e92811a472021bf6bce (127.17.44.193:43297)
I20260812 11:09:44.067234 22573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.193@KRBTEST.COM'} at 127.17.44.193:37831
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
I20260812 11:09:44.082468 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:39869
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:43067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:44.203521 22898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:44.203887 22898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:44.203986 22898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:44.208608 22898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:44.208670 22898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:44.208755 22898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:09:44.214432 22898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:43067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:39869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:44.215492 22898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:44.216478 22898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:44.226667 22907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:44.226676 22903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:44.226678 22904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:44.227025 22898 server_base.cc:1034] running on GCE node
I20260812 11:09:44.227376 22898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:44.227979 22898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:44.229148 22898 hybrid_clock.cc:648] HybridClock initialized: now 1786532984229124 us; error 35 us; skew 500 ppm
Aug 12 11:09:44 dist-test-slave-drl0 krb5kdc[22485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532984, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:44.232134 22898 init.cc:377] Logged in from keytab as kudu/127.17.44.194@KRBTEST.COM (short username kudu)
I20260812 11:09:44.233059 22898 webserver.cc:460] Webserver started at http://127.17.44.194:44677/ using document root <none> and password file <none>
I20260812 11:09:44.233618 22898 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:44.233708 22898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:44.233938 22898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:44.236063 22898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "aeda265230b04dc8b5598e553f74d50c"
format_stamp: "Formatted at 2026-08-12 11:09:44 on dist-test-slave-drl0"
server_key: "b3e9e9e4608a6be5931f778be0c1528e"
server_key_iv: "5d2b62182abef708a2d0dc82837f9c27"
server_key_version: "encryptionkey@0"
I20260812 11:09:44.236632 22898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "aeda265230b04dc8b5598e553f74d50c"
format_stamp: "Formatted at 2026-08-12 11:09:44 on dist-test-slave-drl0"
server_key: "b3e9e9e4608a6be5931f778be0c1528e"
server_key_iv: "5d2b62182abef708a2d0dc82837f9c27"
server_key_version: "encryptionkey@0"
I20260812 11:09:44.240912 22898 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 11:09:44.243758 22917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:44.245177 22898 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:09:44.245334 22898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "aeda265230b04dc8b5598e553f74d50c"
format_stamp: "Formatted at 2026-08-12 11:09:44 on dist-test-slave-drl0"
server_key: "b3e9e9e4608a6be5931f778be0c1528e"
server_key_iv: "5d2b62182abef708a2d0dc82837f9c27"
server_key_version: "encryptionkey@0"
I20260812 11:09:44.245456 22898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:44.258482 22898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:44.262529 22898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:44.262871 22898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:44.263556 22898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:44.264928 22898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:44.265069 22898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:44.265159 22898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:44.265204 22898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:44.275898 22898 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:45003
I20260812 11:09:44.275965 23101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:45003 every 8 connection(s)
I20260812 11:09:44.276831 22898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:09:44.280140 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 22898
I20260812 11:09:44.280262 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:09:44.280551 17587 external_mini_cluster.cc:1428] Setting key 99c3c3ce4aa041cfb9355da1caeb78a4
Aug 12 11:09:44 dist-test-slave-drl0 krb5kdc[22485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532984, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.44.195@KRBTEST.COM; defaulting to no policy
I20260812 11:09:44.292140 22636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:44.278705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:44879 (local address 127.17.44.254:39869)
0812 11:09:44.278871 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:44.278874 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:44.279763 (+ 889us) server_negotiation.cc:407] Connection header received
0812 11:09:44.281351 (+ 1588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:44.281355 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:44.281413 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:44.281522 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:44.282839 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:44.283435 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:44.285149 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:44.285372 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:44.285458 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:44.288159 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:44.288162 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:44.288165 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:44.288205 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:44.289975 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:44.290512 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:44.290516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:44.290518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:44.290601 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:44.290968 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:44.290971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:44.290972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:44.291156 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:44.291269 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:44.291817 (+ 548us) server_negotiation.cc:299] Negotiation successful
0812 11:09:44.291951 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 11:09:44.293205 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:44.279095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:39869 (local address 127.17.44.194:44879)
0812 11:09:44.279591 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:44.279639 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:44.281136 (+ 1497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:44.281694 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:44.281701 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:44.282186 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:44.282673 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:44.282686 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:44.283569 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:44.283572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:44.284981 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:44.284996 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:44.285509 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:44.285512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:44.285689 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:44.286293 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:44.286316 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:44.287887 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:44.290110 (+ 2223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:44.290116 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:44.290131 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:44.290403 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:44.290693 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:44.290697 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:44.290698 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:44.290842 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:44.291275 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:44.291284 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:44.291583 (+ 299us) client_negotiation.cc:769] Sending connection context
0812 11:09:44.291775 (+ 192us) client_negotiation.cc:240] Negotiation successful
0812 11:09:44.292093 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":104,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.195@KRBTEST.COM" created.
I20260812 11:09:44.294507 23102 heartbeater.cc:347] Connected to a master server at 127.17.44.254:39869
I20260812 11:09:44.294770 23102 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:44.295404 23102 heartbeater.cc:511] Master 127.17.44.254:39869 requested a full tablet report, sending...
I20260812 11:09:44.296916 22573 ts_manager.cc:194] Registered new tserver with Master: aeda265230b04dc8b5598e553f74d50c (127.17.44.194:45003)
I20260812 11:09:44.297968 22573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.194@KRBTEST.COM'} at 127.17.44.194:44879
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
I20260812 11:09:44.314653 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:39869
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:43067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:44.439760 23114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:44.440062 23114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:44.440150 23114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:44.444433 23114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:44.444497 23114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:44.444602 23114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:09:44.450304 23114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:43067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:39869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:44.451474 23114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:44.452509 23114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:44.462496 23121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:44.462563 23123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:44.462520 23120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:44.462677 23114 server_base.cc:1034] running on GCE node
I20260812 11:09:44.463323 23114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:44.463946 23114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:44.465130 23114 hybrid_clock.cc:648] HybridClock initialized: now 1786532984465103 us; error 41 us; skew 500 ppm
Aug 12 11:09:44 dist-test-slave-drl0 krb5kdc[22485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532984, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:44.468696 23114 init.cc:377] Logged in from keytab as kudu/127.17.44.195@KRBTEST.COM (short username kudu)
I20260812 11:09:44.470048 23114 webserver.cc:460] Webserver started at http://127.17.44.195:41105/ using document root <none> and password file <none>
I20260812 11:09:44.471016 23114 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:44.471266 23114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:44.471594 23114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:44.473711 23114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "f9359c790091426d8c0a4e46c200a1fb"
format_stamp: "Formatted at 2026-08-12 11:09:44 on dist-test-slave-drl0"
server_key: "9fdc4aa566dba74190ea2f38426be225"
server_key_iv: "36ac6a32b47445c5868e53890fa3a6e5"
server_key_version: "encryptionkey@0"
I20260812 11:09:44.474289 23114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "f9359c790091426d8c0a4e46c200a1fb"
format_stamp: "Formatted at 2026-08-12 11:09:44 on dist-test-slave-drl0"
server_key: "9fdc4aa566dba74190ea2f38426be225"
server_key_iv: "36ac6a32b47445c5868e53890fa3a6e5"
server_key_version: "encryptionkey@0"
I20260812 11:09:44.479821 23114 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260812 11:09:44.482533 23132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:44.484184 23114 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 11:09:44.484342 23114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "f9359c790091426d8c0a4e46c200a1fb"
format_stamp: "Formatted at 2026-08-12 11:09:44 on dist-test-slave-drl0"
server_key: "9fdc4aa566dba74190ea2f38426be225"
server_key_iv: "36ac6a32b47445c5868e53890fa3a6e5"
server_key_version: "encryptionkey@0"
I20260812 11:09:44.484458 23114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:44.521998 23114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:44.525503 23114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:44.525722 23114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:44.526367 23114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:44.527283 23114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:44.527350 23114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:44.527415 23114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:44.527457 23114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:44.539539 23114 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:39965
I20260812 11:09:44.539625 23304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:39965 every 8 connection(s)
I20260812 11:09:44.540474 23114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:09:44.542560 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 23114
I20260812 11:09:44.542713 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:09:44.543197 17587 external_mini_cluster.cc:1428] Setting key b5f6608f4cf18d6bbac005126841c80f
Aug 12 11:09:44 dist-test-slave-drl0 krb5kdc[22485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532984, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:44.557343 22636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:44.543316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:50355 (local address 127.17.44.254:39869)
0812 11:09:44.543497 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:44.543502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:44.544764 (+ 1262us) server_negotiation.cc:407] Connection header received
0812 11:09:44.546380 (+ 1616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:44.546383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:44.546440 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:44.546522 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:44.547755 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:44.548305 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:44.549493 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:44.549727 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:44.549835 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:44.552413 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:44.552416 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:44.552419 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:44.552449 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:44.554491 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:44.555126 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:44.555130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:44.555132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:44.555273 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:44.555730 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:44.555734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:44.555761 (+ 27us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:44.555940 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:44.556065 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:44.556822 (+ 757us) server_negotiation.cc:299] Negotiation successful
0812 11:09:44.557092 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 11:09:44.557922 23308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:44.543688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:39869 (local address 127.17.44.195:50355)
0812 11:09:44.544459 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:44.544609 (+ 150us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:44.546172 (+ 1563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:44.546688 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:44.546695 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:44.547116 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:44.547608 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:44.547622 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:44.548437 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:44.548440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:44.549361 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:44.549371 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:44.549874 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:44.549878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:44.550081 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:44.550682 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:44.550699 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:44.552243 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:44.554642 (+ 2399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:44.554654 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:44.554666 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:44.555008 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:44.555400 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:44.555403 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:44.555405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:44.555624 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:44.556065 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:44.556072 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:44.556333 (+ 261us) client_negotiation.cc:769] Sending connection context
0812 11:09:44.556533 (+ 200us) client_negotiation.cc:240] Negotiation successful
0812 11:09:44.556869 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":166,"threads_started":1}
I20260812 11:09:44.559273 23305 heartbeater.cc:347] Connected to a master server at 127.17.44.254:39869
I20260812 11:09:44.559723 23305 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:44.560462 23305 heartbeater.cc:511] Master 127.17.44.254:39869 requested a full tablet report, sending...
I20260812 11:09:44.561877 22573 ts_manager.cc:194] Registered new tserver with Master: f9359c790091426d8c0a4e46c200a1fb (127.17.44.195:39965)
I20260812 11:09:44.562486 22573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.195@KRBTEST.COM'} at 127.17.44.195:50355
I20260812 11:09:44.570775 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 11:09:44 dist-test-slave-drl0 krb5kdc[22485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532984, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 11:09:44 dist-test-slave-drl0 krb5kdc[22485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532984, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:44.602401 22636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:44.591958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36324 (local address 127.17.44.254:39869)
0812 11:09:44.592209 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:44.592214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:44.592354 (+ 140us) server_negotiation.cc:407] Connection header received
0812 11:09:44.592527 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:44.592530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:44.592573 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:44.592655 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:44.593892 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:44.594509 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:44.595720 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:44.595971 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:44.596074 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:44.598619 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:44.598622 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:44.598625 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:44.598648 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:44.600337 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:44.600782 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:44.600785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:44.600786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:44.600867 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:44.601167 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:44.601170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:44.601171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:44.601409 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:44.601517 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:44.601970 (+ 453us) server_negotiation.cc:299] Negotiation successful
0812 11:09:44.602250 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
W20260812 11:09:44.602870 22573 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36324
I20260812 11:09:44.603974 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 22668
I20260812 11:09:44.621122 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 22898
I20260812 11:09:44.641459 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 23114
I20260812 11:09:44.677768 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 22507
2026-08-12T11:09:44Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2623 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:09:44 dist-test-slave-drl0 krb5kdc[23326](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:09:44 dist-test-slave-drl0 krb5kdc[23326](info): set up 2 sockets
Aug 12 11:09:44 dist-test-slave-drl0 krb5kdc[23326](info): commencing operation
krb5kdc: starting...
W20260812 11:09:45.984787 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.277s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:09:46 dist-test-slave-drl0 krb5kdc[23326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:09:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:09:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:09:46.127228 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:35697
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:35697
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:46.248212 23352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:46.248821 23352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:46.248876 23352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:46.254400 23352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:09:46.254485 23352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:46.254508 23352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:46.254526 23352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:09:46.254544 23352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:09:46.260705 23352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:35697
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:35697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:46.261705 23352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:46.262646 23352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:46.271214 23364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:46.271247 23362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:46.271373 23352 server_base.cc:1034] running on GCE node
W20260812 11:09:46.271310 23361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:46.272003 23352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:46.273069 23352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:46.274356 23352 hybrid_clock.cc:648] HybridClock initialized: now 1786532986274308 us; error 48 us; skew 500 ppm
Aug 12 11:09:46 dist-test-slave-drl0 krb5kdc[23326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532986, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:46.278185 23352 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:09:46.279543 23352 webserver.cc:460] Webserver started at http://127.17.44.254:40625/ using document root <none> and password file <none>
I20260812 11:09:46.280375 23352 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:46.280509 23352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:46.280766 23352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:46.282835 23352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "20a9d7db97a943299a41517377b7c3d6"
format_stamp: "Formatted at 2026-08-12 11:09:46 on dist-test-slave-drl0"
server_key: "89761e03d7d9e98513bf99b08b646ec2"
server_key_iv: "eaf6fc3047663a34f45bdf6e7a98e1da"
server_key_version: "encryptionkey@0"
I20260812 11:09:46.283494 23352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "20a9d7db97a943299a41517377b7c3d6"
format_stamp: "Formatted at 2026-08-12 11:09:46 on dist-test-slave-drl0"
server_key: "89761e03d7d9e98513bf99b08b646ec2"
server_key_iv: "eaf6fc3047663a34f45bdf6e7a98e1da"
server_key_version: "encryptionkey@0"
I20260812 11:09:46.287376 23352 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 11:09:46.290112 23378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:46.291641 23352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:09:46.291805 23352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "20a9d7db97a943299a41517377b7c3d6"
format_stamp: "Formatted at 2026-08-12 11:09:46 on dist-test-slave-drl0"
server_key: "89761e03d7d9e98513bf99b08b646ec2"
server_key_iv: "eaf6fc3047663a34f45bdf6e7a98e1da"
server_key_version: "encryptionkey@0"
I20260812 11:09:46.291926 23352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:46.319063 23352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:46.322566 23352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:46.322783 23352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:46.330902 23352 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:35697
I20260812 11:09:46.331244 23462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:35697 every 8 connection(s)
I20260812 11:09:46.331955 23352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:09:46.334828 23463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:46.335131 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 23352
I20260812 11:09:46.335270 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:09:46.335755 17587 external_mini_cluster.cc:1428] Setting key a35c3429fdf3c3af3995b39aa14e44e8
I20260812 11:09:46.343245 23463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20a9d7db97a943299a41517377b7c3d6: Bootstrap starting.
I20260812 11:09:46.346131 23463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20a9d7db97a943299a41517377b7c3d6: Neither blocks nor log segments found. Creating new log.
Aug 12 11:09:46 dist-test-slave-drl0 krb5kdc[23326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:46.347153 23463 log.cc:824] T 00000000000000000000000000000000 P 20a9d7db97a943299a41517377b7c3d6: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:46.349174 23463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20a9d7db97a943299a41517377b7c3d6: No bootstrap required, opened a new log
I20260812 11:09:46.353564 23468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:46.338117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49626 (local address 127.17.44.254:35697)
0812 11:09:46.338637 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:46.338666 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:46.338714 (+ 48us) server_negotiation.cc:407] Connection header received
0812 11:09:46.339441 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:46.339460 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:46.339794 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:46.340085 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:46.341465 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:46.342485 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:46.343773 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:46.344432 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:46.344572 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:46.347649 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:46.347654 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:46.347667 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:46.347693 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:46.349610 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:46.350148 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:46.350153 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:46.350158 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:46.350282 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:46.350631 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:46.350634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:46.350635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:46.351143 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:46.351373 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:46.352271 (+ 898us) server_negotiation.cc:299] Negotiation successful
0812 11:09:46.352679 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":142,"threads_started":1}
I20260812 11:09:46.353797 23463 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20a9d7db97a943299a41517377b7c3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a9d7db97a943299a41517377b7c3d6" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 35697 } }
I20260812 11:09:46.354053 23463 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20a9d7db97a943299a41517377b7c3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:46.354146 23463 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20a9d7db97a943299a41517377b7c3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20a9d7db97a943299a41517377b7c3d6, State: Initialized, Role: FOLLOWER
I20260812 11:09:46.354666 23463 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20a9d7db97a943299a41517377b7c3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a9d7db97a943299a41517377b7c3d6" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 35697 } }
I20260812 11:09:46.354839 23463 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20a9d7db97a943299a41517377b7c3d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:09:46.354905 23463 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20a9d7db97a943299a41517377b7c3d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:09:46.355029 23463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20a9d7db97a943299a41517377b7c3d6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:46.356297 23463 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20a9d7db97a943299a41517377b7c3d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a9d7db97a943299a41517377b7c3d6" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 35697 } }
I20260812 11:09:46.356595 23463 leader_election.cc:302] T 00000000000000000000000000000000 P 20a9d7db97a943299a41517377b7c3d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20a9d7db97a943299a41517377b7c3d6; no voters:
I20260812 11:09:46.356880 23463 leader_election.cc:288] T 00000000000000000000000000000000 P 20a9d7db97a943299a41517377b7c3d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:09:46.357008 23470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20a9d7db97a943299a41517377b7c3d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:46.357304 23470 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20a9d7db97a943299a41517377b7c3d6 [term 1 LEADER]: Becoming Leader. State: Replica: 20a9d7db97a943299a41517377b7c3d6, State: Running, Role: LEADER
I20260812 11:09:46.357673 23470 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20a9d7db97a943299a41517377b7c3d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a9d7db97a943299a41517377b7c3d6" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 35697 } }
I20260812 11:09:46.358170 23463 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20a9d7db97a943299a41517377b7c3d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:09:46.359407 23472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20a9d7db97a943299a41517377b7c3d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20a9d7db97a943299a41517377b7c3d6. Latest consensus state: current_term: 1 leader_uuid: "20a9d7db97a943299a41517377b7c3d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a9d7db97a943299a41517377b7c3d6" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 35697 } } }
I20260812 11:09:46.359555 23472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20a9d7db97a943299a41517377b7c3d6 [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:46.360006 23481 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:09:46.359972 23471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20a9d7db97a943299a41517377b7c3d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20a9d7db97a943299a41517377b7c3d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a9d7db97a943299a41517377b7c3d6" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 35697 } } }
I20260812 11:09:46.360174 23471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20a9d7db97a943299a41517377b7c3d6 [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:46.364228 23481 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:09:46.370791 23481 catalog_manager.cc:1347] Generated new cluster ID: da29e753f7cf4996bf55414ec367b417
I20260812 11:09:46.370903 23481 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:09:46.381016 23481 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:09:46.382627 23481 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:09:46.395715 23481 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20a9d7db97a943299a41517377b7c3d6: Generated new TSK 0
I20260812 11:09:46.396468 23481 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
I20260812 11:09:46.430745 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:35697
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:46.551144 23499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:46.551405 23499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:46.551448 23499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:46.555557 23499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:46.555611 23499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:46.555696 23499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:09:46.561434 23499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:35697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:46.562496 23499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:46.563637 23499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:46.573508 23511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:46.573508 23509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:46.573597 23508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:46.573954 23499 server_base.cc:1034] running on GCE node
I20260812 11:09:46.574431 23499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:46.575201 23499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:46.576395 23499 hybrid_clock.cc:648] HybridClock initialized: now 1786532986576380 us; error 46 us; skew 500 ppm
Aug 12 11:09:46 dist-test-slave-drl0 krb5kdc[23326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532986, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:46.580168 23499 init.cc:377] Logged in from keytab as kudu/127.17.44.193@KRBTEST.COM (short username kudu)
I20260812 11:09:46.581103 23499 webserver.cc:460] Webserver started at http://127.17.44.193:36375/ using document root <none> and password file <none>
I20260812 11:09:46.581641 23499 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:46.581702 23499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:46.581872 23499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:46.583936 23499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "a5852eaf192448fcad4093e04c466644"
format_stamp: "Formatted at 2026-08-12 11:09:46 on dist-test-slave-drl0"
server_key: "4c29bb1e034b52befa9af6b1f11d5fd9"
server_key_iv: "f2122197b032401561589634cf428907"
server_key_version: "encryptionkey@0"
I20260812 11:09:46.584455 23499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "a5852eaf192448fcad4093e04c466644"
format_stamp: "Formatted at 2026-08-12 11:09:46 on dist-test-slave-drl0"
server_key: "4c29bb1e034b52befa9af6b1f11d5fd9"
server_key_iv: "f2122197b032401561589634cf428907"
server_key_version: "encryptionkey@0"
I20260812 11:09:46.589444 23499 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 11:09:46.592141 23521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:46.593669 23499 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 11:09:46.593828 23499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "a5852eaf192448fcad4093e04c466644"
format_stamp: "Formatted at 2026-08-12 11:09:46 on dist-test-slave-drl0"
server_key: "4c29bb1e034b52befa9af6b1f11d5fd9"
server_key_iv: "f2122197b032401561589634cf428907"
server_key_version: "encryptionkey@0"
I20260812 11:09:46.593945 23499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:46.629240 23499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:46.632848 23499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:46.633077 23499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:46.633765 23499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:46.634660 23499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:46.634738 23499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:46.634814 23499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:46.634852 23499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:46.646482 23499 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:34375
I20260812 11:09:46.646513 23699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:34375 every 8 connection(s)
I20260812 11:09:46.647576 23499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:09:46.651015 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 23499
I20260812 11:09:46.651218 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:09:46.651691 17587 external_mini_cluster.cc:1428] Setting key 6603913429617894d0b0dc9bdb3775f3
Aug 12 11:09:46 dist-test-slave-drl0 krb5kdc[23326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532986, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.44.194@KRBTEST.COM; defaulting to no policy
I20260812 11:09:46.665696 23468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:46.650799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:48149 (local address 127.17.44.254:35697)
0812 11:09:46.651117 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:46.651124 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:46.652337 (+ 1213us) server_negotiation.cc:407] Connection header received
0812 11:09:46.654126 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:46.654128 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:46.654174 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:46.654247 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:46.655954 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:46.656832 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:46.658063 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:46.658289 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:46.658432 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:46.661534 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:46.661538 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:46.661541 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:46.661570 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:46.663128 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:46.663670 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:46.663681 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:46.663687 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:46.663772 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:46.664267 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:46.664270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:46.664272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:46.664713 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:46.664847 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:46.665379 (+ 532us) server_negotiation.cc:299] Negotiation successful
0812 11:09:46.665516 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.194@KRBTEST.COM" created.
I20260812 11:09:46.666729 23702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:46.651222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:35697 (local address 127.17.44.193:48149)
0812 11:09:46.651995 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:46.652092 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:46.653867 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:46.654434 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:46.654446 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:46.655043 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:46.655766 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:46.655787 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:46.656984 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:46.656987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:46.657928 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:46.657938 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:46.658473 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:46.658478 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:46.658717 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:46.659473 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:46.659515 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:46.661346 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:46.663252 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:46.663258 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:46.663273 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:46.663570 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:46.663870 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:46.663873 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:46.663875 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:46.664128 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:46.664856 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:46.664865 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:46.665125 (+ 260us) client_negotiation.cc:769] Sending connection context
0812 11:09:46.665319 (+ 194us) client_negotiation.cc:240] Negotiation successful
0812 11:09:46.665667 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":130,"threads_started":1}
I20260812 11:09:46.668180 23700 heartbeater.cc:347] Connected to a master server at 127.17.44.254:35697
I20260812 11:09:46.668467 23700 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:46.668999 23700 heartbeater.cc:511] Master 127.17.44.254:35697 requested a full tablet report, sending...
I20260812 11:09:46.670899 23401 ts_manager.cc:194] Registered new tserver with Master: a5852eaf192448fcad4093e04c466644 (127.17.44.193:34375)
I20260812 11:09:46.672200 23401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.193@KRBTEST.COM'} at 127.17.44.193:48149
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
I20260812 11:09:46.684006 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:35697
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:46.806500 23708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:46.806764 23708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:46.806808 23708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:46.810837 23708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:46.810890 23708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:46.811017 23708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:09:46.816042 23708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:35697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:46.816982 23708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:46.817929 23708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:46.827884 23722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:46.827936 23718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:46.827880 23713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:46.828254 23708 server_base.cc:1034] running on GCE node
I20260812 11:09:46.828622 23708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:46.829228 23708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:46.830420 23708 hybrid_clock.cc:648] HybridClock initialized: now 1786532986830394 us; error 39 us; skew 500 ppm
Aug 12 11:09:46 dist-test-slave-drl0 krb5kdc[23326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532986, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:46.834267 23708 init.cc:377] Logged in from keytab as kudu/127.17.44.194@KRBTEST.COM (short username kudu)
I20260812 11:09:46.835721 23708 webserver.cc:460] Webserver started at http://127.17.44.194:38725/ using document root <none> and password file <none>
I20260812 11:09:46.836493 23708 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:46.836599 23708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:46.836889 23708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:46.839742 23708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "3c73e89abe8c49178c7c98ccd74113f1"
format_stamp: "Formatted at 2026-08-12 11:09:46 on dist-test-slave-drl0"
server_key: "a96e8361b2bbf8a9483015c1cdd7d28a"
server_key_iv: "9eac425bda9eedb2b3dfbbd42ff34112"
server_key_version: "encryptionkey@0"
I20260812 11:09:46.840482 23708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "3c73e89abe8c49178c7c98ccd74113f1"
format_stamp: "Formatted at 2026-08-12 11:09:46 on dist-test-slave-drl0"
server_key: "a96e8361b2bbf8a9483015c1cdd7d28a"
server_key_iv: "9eac425bda9eedb2b3dfbbd42ff34112"
server_key_version: "encryptionkey@0"
I20260812 11:09:46.844449 23708 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 11:09:46.847673 23733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:46.849001 23708 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:09:46.849154 23708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "3c73e89abe8c49178c7c98ccd74113f1"
format_stamp: "Formatted at 2026-08-12 11:09:46 on dist-test-slave-drl0"
server_key: "a96e8361b2bbf8a9483015c1cdd7d28a"
server_key_iv: "9eac425bda9eedb2b3dfbbd42ff34112"
server_key_version: "encryptionkey@0"
I20260812 11:09:46.849277 23708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:46.876284 23708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:46.879649 23708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:46.879863 23708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:46.880488 23708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:46.881376 23708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:46.881449 23708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:46.881521 23708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:46.881574 23708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:46.891286 23708 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:42145
I20260812 11:09:46.891341 23930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:42145 every 8 connection(s)
I20260812 11:09:46.892202 23708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:09:46.901584 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 23708
I20260812 11:09:46.901713 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:09:46.902036 17587 external_mini_cluster.cc:1428] Setting key 8344a94b9891d283621a3febe7fdf8a0
Aug 12 11:09:46 dist-test-slave-drl0 krb5kdc[23326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532986, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:46.909850 23468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:46.894639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:52977 (local address 127.17.44.254:35697)
0812 11:09:46.894854 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:46.894860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:46.896314 (+ 1454us) server_negotiation.cc:407] Connection header received
0812 11:09:46.897894 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:46.897897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:46.897947 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:46.898033 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:46.899337 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:46.899866 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:46.901396 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:46.901733 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:46.901872 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:46.904696 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:46.904703 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:46.904709 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:46.904744 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:46.906626 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:46.907409 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:46.907412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:46.907413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:46.907494 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:46.908287 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:46.908291 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:46.908293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:46.908655 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:46.908753 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:46.909519 (+ 766us) server_negotiation.cc:299] Negotiation successful
0812 11:09:46.909623 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 11:09:46.911002 23934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:46.895317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:35697 (local address 127.17.44.194:52977)
0812 11:09:46.896128 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:46.896174 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:46.897593 (+ 1419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:46.898189 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:46.898196 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:46.898551 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:46.899167 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:46.899183 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:46.900046 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:46.900049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:46.901229 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:46.901240 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:46.901922 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:46.901926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:46.902077 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:46.902754 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:46.902777 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:46.904524 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:46.906783 (+ 2259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:46.906791 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:46.906810 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:46.907257 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:46.907598 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:46.907604 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:46.907609 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:46.908081 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:46.908816 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:46.908826 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:46.909178 (+ 352us) client_negotiation.cc:769] Sending connection context
0812 11:09:46.909443 (+ 265us) client_negotiation.cc:240] Negotiation successful
0812 11:09:46.909827 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":205,"threads_started":1}
I20260812 11:09:46.912269 23932 heartbeater.cc:347] Connected to a master server at 127.17.44.254:35697
I20260812 11:09:46.912734 23932 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:46.913362 23932 heartbeater.cc:511] Master 127.17.44.254:35697 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.44.195@KRBTEST.COM; defaulting to no policy
I20260812 11:09:46.915072 23401 ts_manager.cc:194] Registered new tserver with Master: 3c73e89abe8c49178c7c98ccd74113f1 (127.17.44.194:42145)
I20260812 11:09:46.915961 23401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.194@KRBTEST.COM'} at 127.17.44.194:52977
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
I20260812 11:09:46.936213 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:35697
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:47.064740 23938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:47.065038 23938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:47.065116 23938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:47.069397 23938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:47.069476 23938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:47.069559 23938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:09:47.075773 23938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:35697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:47.076872 23938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:47.078053 23938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:47.088201 23945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:47.088212 23947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:47.088212 23944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:47.088549 23938 server_base.cc:1034] running on GCE node
I20260812 11:09:47.089074 23938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:47.089761 23938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:47.091017 23938 hybrid_clock.cc:648] HybridClock initialized: now 1786532987090991 us; error 33 us; skew 500 ppm
Aug 12 11:09:47 dist-test-slave-drl0 krb5kdc[23326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532987, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:47.094004 23938 init.cc:377] Logged in from keytab as kudu/127.17.44.195@KRBTEST.COM (short username kudu)
I20260812 11:09:47.094892 23938 webserver.cc:460] Webserver started at http://127.17.44.195:46379/ using document root <none> and password file <none>
I20260812 11:09:47.095537 23938 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:47.095600 23938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:47.095822 23938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:47.098001 23938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "23e173d8da08411cb153027040945aee"
format_stamp: "Formatted at 2026-08-12 11:09:47 on dist-test-slave-drl0"
server_key: "0cc46de8ca4de2debd70fe4535466b20"
server_key_iv: "251e9eca1bf0f81f6a6b9468823890ce"
server_key_version: "encryptionkey@0"
I20260812 11:09:47.098673 23938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "23e173d8da08411cb153027040945aee"
format_stamp: "Formatted at 2026-08-12 11:09:47 on dist-test-slave-drl0"
server_key: "0cc46de8ca4de2debd70fe4535466b20"
server_key_iv: "251e9eca1bf0f81f6a6b9468823890ce"
server_key_version: "encryptionkey@0"
I20260812 11:09:47.103264 23938 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260812 11:09:47.106043 23959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:47.107689 23938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:09:47.107832 23938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "23e173d8da08411cb153027040945aee"
format_stamp: "Formatted at 2026-08-12 11:09:47 on dist-test-slave-drl0"
server_key: "0cc46de8ca4de2debd70fe4535466b20"
server_key_iv: "251e9eca1bf0f81f6a6b9468823890ce"
server_key_version: "encryptionkey@0"
I20260812 11:09:47.107951 23938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:47.136937 23938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:47.140352 23938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:47.140563 23938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:47.141348 23938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:47.142297 23938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:47.142375 23938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:47.142458 23938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:47.142511 23938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:47.153299 23938 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:35171
I20260812 11:09:47.153335 24149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:35171 every 8 connection(s)
I20260812 11:09:47.154201 23938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:09:47.154910 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 23938
I20260812 11:09:47.155067 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:09:47.155378 17587 external_mini_cluster.cc:1428] Setting key 26ee47c2e067c8f4975ad46f1f6c410a
Aug 12 11:09:47 dist-test-slave-drl0 krb5kdc[23326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532987, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:47.171913 23468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:47.156569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:41241 (local address 127.17.44.254:35697)
0812 11:09:47.156910 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:47.156915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:47.157801 (+ 886us) server_negotiation.cc:407] Connection header received
0812 11:09:47.159552 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:47.159555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:47.159606 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:47.159686 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:47.161136 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:47.161681 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:47.163170 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:47.163394 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:47.163620 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:47.167302 (+ 3682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:47.167306 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:47.167308 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:47.167339 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:47.169006 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:47.169978 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:47.169983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:47.169985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:47.170153 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:47.170683 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:47.170686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:47.170687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:47.170891 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:47.171083 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:47.171613 (+ 530us) server_negotiation.cc:299] Negotiation successful
0812 11:09:47.171746 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241}
I20260812 11:09:47.172842 24155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:47.157111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:35697 (local address 127.17.44.195:41241)
0812 11:09:47.157604 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:47.157657 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:47.159362 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:47.159975 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:47.159983 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:47.160441 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:47.160960 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:47.160976 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:47.161835 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:47.161838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:47.163027 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:47.163038 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:47.163730 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:47.163735 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:47.163991 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:47.164907 (+ 916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:47.164928 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:47.167054 (+ 2126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:47.169150 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:47.169164 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:47.169178 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:47.169773 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:47.170297 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:47.170300 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:47.170302 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:47.170558 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:47.171084 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:47.171091 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:47.171358 (+ 267us) client_negotiation.cc:769] Sending connection context
0812 11:09:47.171552 (+ 194us) client_negotiation.cc:240] Negotiation successful
0812 11:09:47.171912 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":145,"threads_started":1}
I20260812 11:09:47.174098 24151 heartbeater.cc:347] Connected to a master server at 127.17.44.254:35697
I20260812 11:09:47.174386 24151 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:47.175032 24151 heartbeater.cc:511] Master 127.17.44.254:35697 requested a full tablet report, sending...
I20260812 11:09:47.176263 23401 ts_manager.cc:194] Registered new tserver with Master: 23e173d8da08411cb153027040945aee (127.17.44.195:35171)
I20260812 11:09:47.176852 23401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.195@KRBTEST.COM'} at 127.17.44.195:41241
I20260812 11:09:47.182643 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 11:09:47 dist-test-slave-drl0 krb5kdc[23326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 11:09:47 dist-test-slave-drl0 krb5kdc[23326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.193@KRBTEST.COM
I20260812 11:09:47.216018 24162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:47.203822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35538 (local address 127.17.44.193:34375)
0812 11:09:47.204172 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:47.204179 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:47.204206 (+ 27us) server_negotiation.cc:407] Connection header received
0812 11:09:47.204260 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:47.204265 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:47.204455 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:47.204585 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:47.205296 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:47.206078 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:47.207528 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:47.208000 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:47.208109 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:47.211134 (+ 3025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:47.211143 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:47.211157 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:47.211181 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:47.213435 (+ 2254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:47.213924 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:47.213929 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:47.213930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:47.214018 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:47.214415 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:47.214420 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:47.214422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:47.214845 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:47.215321 (+ 476us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:47.215706 (+ 385us) server_negotiation.cc:299] Negotiation successful
0812 11:09:47.215872 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":115,"threads_started":1}
Aug 12 11:09:47 dist-test-slave-drl0 krb5kdc[23326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:47.230049 23468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:47.219531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49628 (local address 127.17.44.254:35697)
0812 11:09:47.219719 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:47.219726 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:47.219793 (+ 67us) server_negotiation.cc:407] Connection header received
0812 11:09:47.219984 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:47.219987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:47.220048 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:47.220151 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:47.220893 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:47.221510 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:47.222561 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:47.222815 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:47.222991 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:47.226061 (+ 3070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:47.226064 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:47.226067 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:47.226092 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:47.227844 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:47.228500 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:47.228504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:47.228505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:47.228609 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:47.229031 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:47.229035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:47.229036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:47.229277 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:47.229417 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:47.229801 (+ 384us) server_negotiation.cc:299] Negotiation successful
0812 11:09:47.229923 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260812 11:09:47.230685 23401 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49628
I20260812 11:09:47.231680 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 23499
I20260812 11:09:47.260159 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 23708
I20260812 11:09:47.270063 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 23938
I20260812 11:09:47.287180 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 23352
2026-08-12T11:09:47Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2614 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:09:47 dist-test-slave-drl0 krb5kdc[24168](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:09:47 dist-test-slave-drl0 krb5kdc[24168](info): set up 2 sockets
Aug 12 11:09:47 dist-test-slave-drl0 krb5kdc[24168](info): commencing operation
krb5kdc: starting...
W20260812 11:09:48.637919 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.315s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260812 11:09:48.794646 17587 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.157s user 0.000s sys 0.003s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260812 11:09:48.968415 17587 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.174s user 0.002s sys 0.000s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260812 11:09:49.165158 17587 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.197s user 0.001s sys 0.001s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/test-user.keytab.
W20260812 11:09:49.704543 17587 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.539s user 0.000s sys 0.002s
Aug 12 11:09:49 dist-test-slave-drl0 krb5kdc[24168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:09:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:09:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
W20260812 11:09:49.866240 17587 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.17.44.254: real 0.135s user 0.000s sys 0.004s
I20260812 11:09:49.867101 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:36343
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:41893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:36343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:49.993628 24193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:49.993901 24193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:49.993945 24193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:49.999441 24193 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:09:49.999507 24193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:49.999528 24193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:49.999547 24193 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:09:49.999562 24193 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:09:50.005322 24193 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:41893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:36343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:36343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:50.006381 24193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:50.007433 24193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:50.016992 24200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:50.017084 24193 server_base.cc:1034] running on GCE node
W20260812 11:09:50.016992 24201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:50.017020 24205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:50.017844 24193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:50.018971 24193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:50.020205 24193 hybrid_clock.cc:648] HybridClock initialized: now 1786532990020183 us; error 33 us; skew 500 ppm
Aug 12 11:09:50 dist-test-slave-drl0 krb5kdc[24168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532990, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:50.027953 24193 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:09:50.028699 24193 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:50.028796 24193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:50.029074 24193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:50.031339 24193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "85ef49b0a8904f9eb7c219dc1c64a445"
format_stamp: "Formatted at 2026-08-12 11:09:50 on dist-test-slave-drl0"
server_key: "f24625812d3b9c3c5803686b5e28bd20"
server_key_iv: "aaba5f73163f9ea81b8a90c8ef8fca5a"
server_key_version: "encryptionkey@0"
I20260812 11:09:50.031929 24193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "85ef49b0a8904f9eb7c219dc1c64a445"
format_stamp: "Formatted at 2026-08-12 11:09:50 on dist-test-slave-drl0"
server_key: "f24625812d3b9c3c5803686b5e28bd20"
server_key_iv: "aaba5f73163f9ea81b8a90c8ef8fca5a"
server_key_version: "encryptionkey@0"
I20260812 11:09:50.037330 24193 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260812 11:09:50.040848 24213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:50.042605 24193 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:09:50.042860 24193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "85ef49b0a8904f9eb7c219dc1c64a445"
format_stamp: "Formatted at 2026-08-12 11:09:50 on dist-test-slave-drl0"
server_key: "f24625812d3b9c3c5803686b5e28bd20"
server_key_iv: "aaba5f73163f9ea81b8a90c8ef8fca5a"
server_key_version: "encryptionkey@0"
I20260812 11:09:50.043092 24193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:50.055331 24193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:50.058898 24193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:50.059154 24193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:50.066646 24193 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:36343
I20260812 11:09:50.067028 24297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:36343 every 8 connection(s)
I20260812 11:09:50.067407 24193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:09:50.070448 24301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:50.075340 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 24193
I20260812 11:09:50.075484 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:09:50.075776 17587 external_mini_cluster.cc:1428] Setting key d86c0fab0711b616722942417402970a
I20260812 11:09:50.076738 24301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85ef49b0a8904f9eb7c219dc1c64a445: Bootstrap starting.
I20260812 11:09:50.079327 24301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85ef49b0a8904f9eb7c219dc1c64a445: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:50.080236 24301 log.cc:824] T 00000000000000000000000000000000 P 85ef49b0a8904f9eb7c219dc1c64a445: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:50.082510 24301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85ef49b0a8904f9eb7c219dc1c64a445: No bootstrap required, opened a new log
Aug 12 11:09:50 dist-test-slave-drl0 krb5kdc[24168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:50.086895 24301 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85ef49b0a8904f9eb7c219dc1c64a445 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85ef49b0a8904f9eb7c219dc1c64a445" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36343 } }
I20260812 11:09:50.087280 24301 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85ef49b0a8904f9eb7c219dc1c64a445 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:50.087368 24301 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85ef49b0a8904f9eb7c219dc1c64a445 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85ef49b0a8904f9eb7c219dc1c64a445, State: Initialized, Role: FOLLOWER
I20260812 11:09:50.087854 24301 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85ef49b0a8904f9eb7c219dc1c64a445 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85ef49b0a8904f9eb7c219dc1c64a445" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36343 } }
I20260812 11:09:50.088006 24301 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85ef49b0a8904f9eb7c219dc1c64a445 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:09:50.088078 24301 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85ef49b0a8904f9eb7c219dc1c64a445 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:09:50.088178 24301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85ef49b0a8904f9eb7c219dc1c64a445 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:50.089382 24301 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85ef49b0a8904f9eb7c219dc1c64a445 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85ef49b0a8904f9eb7c219dc1c64a445" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36343 } }
I20260812 11:09:50.089949 24301 leader_election.cc:302] T 00000000000000000000000000000000 P 85ef49b0a8904f9eb7c219dc1c64a445 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85ef49b0a8904f9eb7c219dc1c64a445; no voters:
I20260812 11:09:50.090570 24301 leader_election.cc:288] T 00000000000000000000000000000000 P 85ef49b0a8904f9eb7c219dc1c64a445 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:09:50.090692 24311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85ef49b0a8904f9eb7c219dc1c64a445 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:50.091046 24311 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85ef49b0a8904f9eb7c219dc1c64a445 [term 1 LEADER]: Becoming Leader. State: Replica: 85ef49b0a8904f9eb7c219dc1c64a445, State: Running, Role: LEADER
I20260812 11:09:50.091426 24311 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85ef49b0a8904f9eb7c219dc1c64a445 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85ef49b0a8904f9eb7c219dc1c64a445" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36343 } }
I20260812 11:09:50.092125 24305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:50.077239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45556 (local address 127.17.44.254:36343)
0812 11:09:50.077654 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:50.077670 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:50.077704 (+ 34us) server_negotiation.cc:407] Connection header received
0812 11:09:50.078341 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:50.078357 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:50.078673 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:50.079058 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:50.079997 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.081262 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.082446 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.083051 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.083180 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:50.086094 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:50.086099 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:50.086113 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:50.086141 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:50.087986 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:50.088555 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:50.088561 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:50.088566 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:50.088703 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:50.089041 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:50.089044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:50.089046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:50.089608 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:50.090027 (+ 419us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:50.090329 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 11:09:50.090835 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":141,"threads_started":1}
I20260812 11:09:50.092378 24301 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85ef49b0a8904f9eb7c219dc1c64a445 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:09:50.093926 24312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85ef49b0a8904f9eb7c219dc1c64a445 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85ef49b0a8904f9eb7c219dc1c64a445" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85ef49b0a8904f9eb7c219dc1c64a445" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36343 } } }
I20260812 11:09:50.094067 24312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85ef49b0a8904f9eb7c219dc1c64a445 [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:50.094424 24313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85ef49b0a8904f9eb7c219dc1c64a445 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85ef49b0a8904f9eb7c219dc1c64a445. Latest consensus state: current_term: 1 leader_uuid: "85ef49b0a8904f9eb7c219dc1c64a445" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85ef49b0a8904f9eb7c219dc1c64a445" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36343 } } }
I20260812 11:09:50.094648 24313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85ef49b0a8904f9eb7c219dc1c64a445 [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:50.095070 24325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:09:50.100385 24325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:09:50.106848 24325 catalog_manager.cc:1347] Generated new cluster ID: 3e54f6e8139b4c54b31b5bfc70ba2e27
I20260812 11:09:50.106954 24325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:09:50.117401 24325 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:09:50.118737 24325 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:09:50.135929 24325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85ef49b0a8904f9eb7c219dc1c64a445: Generated new TSK 0
I20260812 11:09:50.136797 24325 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
I20260812 11:09:50.179551 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:36343
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:41893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:50.302201 24338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:50.302450 24338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:50.302493 24338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:50.306581 24338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:50.306638 24338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:50.306696 24338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:09:50.312465 24338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:41893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:36343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:50.313493 24338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:50.314436 24338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:50.324604 24348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:50.324586 24347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:50.324587 24353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:50.324975 24338 server_base.cc:1034] running on GCE node
I20260812 11:09:50.325630 24338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:50.326251 24338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:50.327482 24338 hybrid_clock.cc:648] HybridClock initialized: now 1786532990327467 us; error 26 us; skew 500 ppm
Aug 12 11:09:50 dist-test-slave-drl0 krb5kdc[24168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532990, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:50.331318 24338 init.cc:377] Logged in from keytab as kudu/127.17.44.193@KRBTEST.COM (short username kudu)
I20260812 11:09:50.332326 24338 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:50.332438 24338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:50.332690 24338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:50.334757 24338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "6a061687201b47acbc76fb41dfa192c0"
format_stamp: "Formatted at 2026-08-12 11:09:50 on dist-test-slave-drl0"
server_key: "b22eb68728e7a839ae76ff7f427d517f"
server_key_iv: "bfafad344aaa7b24d7db4fd8a0ef543d"
server_key_version: "encryptionkey@0"
I20260812 11:09:50.335407 24338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "6a061687201b47acbc76fb41dfa192c0"
format_stamp: "Formatted at 2026-08-12 11:09:50 on dist-test-slave-drl0"
server_key: "b22eb68728e7a839ae76ff7f427d517f"
server_key_iv: "bfafad344aaa7b24d7db4fd8a0ef543d"
server_key_version: "encryptionkey@0"
I20260812 11:09:50.339435 24338 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 11:09:50.341915 24363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:50.343343 24338 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:09:50.343498 24338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "6a061687201b47acbc76fb41dfa192c0"
format_stamp: "Formatted at 2026-08-12 11:09:50 on dist-test-slave-drl0"
server_key: "b22eb68728e7a839ae76ff7f427d517f"
server_key_iv: "bfafad344aaa7b24d7db4fd8a0ef543d"
server_key_version: "encryptionkey@0"
I20260812 11:09:50.343616 24338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:50.370702 24338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:50.374115 24338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:50.374452 24338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:50.375120 24338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:50.375981 24338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:50.376056 24338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:50.376150 24338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:50.376194 24338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:50.386916 24338 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:36927
I20260812 11:09:50.387008 24547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:36927 every 8 connection(s)
I20260812 11:09:50.388098 24338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:09:50.397159 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 24338
I20260812 11:09:50.397281 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:09:50.397913 17587 external_mini_cluster.cc:1428] Setting key 98049cad02cd8213845cd55568577b55
Aug 12 11:09:50 dist-test-slave-drl0 krb5kdc[24168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532990, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:50.404829 24305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:50.390287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:43161 (local address 127.17.44.254:36343)
0812 11:09:50.390448 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:50.390452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:50.391361 (+ 909us) server_negotiation.cc:407] Connection header received
0812 11:09:50.393168 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:50.393172 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:50.393262 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:50.393367 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:50.394906 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.395502 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.396658 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.396938 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.397056 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:50.400113 (+ 3057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:50.400117 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:50.400120 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:50.400145 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:50.402044 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:50.402574 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:50.402579 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:50.402582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:50.402683 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:50.403397 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:50.403403 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:50.403405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:50.403665 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:50.403790 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:50.404596 (+ 806us) server_negotiation.cc:299] Negotiation successful
0812 11:09:50.404706 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 11:09:50.406106 24551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:50.390625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36343 (local address 127.17.44.193:43161)
0812 11:09:50.391205 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:50.391247 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:50.392629 (+ 1382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:50.393745 (+ 1116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:50.393752 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:50.394116 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:50.394615 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:50.394628 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.395647 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.395652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:50.396523 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:50.396533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.397112 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.397115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:50.397308 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:50.398136 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:50.398153 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:50.399949 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:50.402186 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:50.402191 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:50.402202 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:50.402467 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:50.402878 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:50.402882 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:50.402884 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:50.403252 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:50.403825 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:50.403831 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:50.404187 (+ 356us) client_negotiation.cc:769] Sending connection context
0812 11:09:50.404433 (+ 246us) client_negotiation.cc:240] Negotiation successful
0812 11:09:50.404895 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":157,"threads_started":1}
I20260812 11:09:50.407724 24548 heartbeater.cc:347] Connected to a master server at 127.17.44.254:36343
I20260812 11:09:50.408037 24548 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:50.408641 24548 heartbeater.cc:511] Master 127.17.44.254:36343 requested a full tablet report, sending...
I20260812 11:09:50.410313 24242 ts_manager.cc:194] Registered new tserver with Master: 6a061687201b47acbc76fb41dfa192c0 (127.17.44.193:36927)
WARNING: no policy specified for kudu/127.17.44.194@KRBTEST.COM; defaulting to no policy
I20260812 11:09:50.411746 24242 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.193@KRBTEST.COM'} at 127.17.44.193:43161
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
I20260812 11:09:50.431399 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:36343
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:41893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:50.553079 24554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:50.553321 24554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:50.553371 24554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:50.557372 24554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:50.557428 24554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:50.557516 24554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:09:50.562814 24554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:41893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:36343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:50.563872 24554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:50.564925 24554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 11:09:50.574812 24554 server_base.cc:1034] running on GCE node
W20260812 11:09:50.574918 24565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:50.575007 24568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:50.574954 24563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:50.575450 24554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:50.576077 24554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:50.577299 24554 hybrid_clock.cc:648] HybridClock initialized: now 1786532990577281 us; error 105 us; skew 500 ppm
Aug 12 11:09:50 dist-test-slave-drl0 krb5kdc[24168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532990, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:50.580216 24554 init.cc:377] Logged in from keytab as kudu/127.17.44.194@KRBTEST.COM (short username kudu)
I20260812 11:09:50.580966 24554 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:50.581063 24554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:50.581315 24554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:50.583464 24554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "337fa4eed860441593796983ecd2f8cd"
format_stamp: "Formatted at 2026-08-12 11:09:50 on dist-test-slave-drl0"
server_key: "6fab529590000f4e8eb89f6b7cc3ea51"
server_key_iv: "7c85b08fbb9ec6a56de6be9b31d56acf"
server_key_version: "encryptionkey@0"
I20260812 11:09:50.584019 24554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "337fa4eed860441593796983ecd2f8cd"
format_stamp: "Formatted at 2026-08-12 11:09:50 on dist-test-slave-drl0"
server_key: "6fab529590000f4e8eb89f6b7cc3ea51"
server_key_iv: "7c85b08fbb9ec6a56de6be9b31d56acf"
server_key_version: "encryptionkey@0"
I20260812 11:09:50.588640 24554 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 11:09:50.592161 24578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:50.593705 24554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:09:50.593883 24554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "337fa4eed860441593796983ecd2f8cd"
format_stamp: "Formatted at 2026-08-12 11:09:50 on dist-test-slave-drl0"
server_key: "6fab529590000f4e8eb89f6b7cc3ea51"
server_key_iv: "7c85b08fbb9ec6a56de6be9b31d56acf"
server_key_version: "encryptionkey@0"
I20260812 11:09:50.594041 24554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:50.604959 24554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:50.608201 24554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:50.608418 24554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:50.609011 24554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:50.609869 24554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:50.609918 24554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:50.609982 24554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:50.610014 24554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:50.620882 24554 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:37697
I20260812 11:09:50.620916 24758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:37697 every 8 connection(s)
I20260812 11:09:50.621538 24554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:09:50.629197 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 24554
I20260812 11:09:50.629330 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:09:50.629637 17587 external_mini_cluster.cc:1428] Setting key 458178bfba2a2564a492b54156e9c07b
Aug 12 11:09:50 dist-test-slave-drl0 krb5kdc[24168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532990, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:50.638917 24305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:50.624147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:57691 (local address 127.17.44.254:36343)
0812 11:09:50.624311 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:50.624315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:50.625106 (+ 791us) server_negotiation.cc:407] Connection header received
0812 11:09:50.626423 (+ 1317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:50.626426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:50.626484 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:50.626599 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:50.628064 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.628606 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.630200 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.630423 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.630537 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:50.633979 (+ 3442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:50.633981 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:50.633983 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:50.634006 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:50.636187 (+ 2181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:50.636931 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:50.636935 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:50.636937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:50.637169 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:50.637614 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:50.637618 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:50.637620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:50.637834 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:50.637954 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:50.638583 (+ 629us) server_negotiation.cc:299] Negotiation successful
0812 11:09:50.638731 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 11:09:50.639750 24766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:50.624277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36343 (local address 127.17.44.194:57691)
0812 11:09:50.624824 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:50.624990 (+ 166us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:50.626215 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:50.626770 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:50.626776 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:50.627269 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:50.627755 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:50.627769 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.628743 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.628746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:50.630072 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:50.630082 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.630580 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.630584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:50.630780 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:50.631572 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:50.631600 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:50.633792 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:50.636413 (+ 2621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:50.636421 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:50.636442 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:50.636802 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:50.637315 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:50.637319 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:50.637328 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:50.637495 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:50.637956 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:50.637964 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:50.638296 (+ 332us) client_negotiation.cc:769] Sending connection context
0812 11:09:50.638514 (+ 218us) client_negotiation.cc:240] Negotiation successful
0812 11:09:50.638861 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":130,"threads_started":1}
I20260812 11:09:50.641211 24759 heartbeater.cc:347] Connected to a master server at 127.17.44.254:36343
I20260812 11:09:50.641492 24759 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:50.641959 24759 heartbeater.cc:511] Master 127.17.44.254:36343 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.44.195@KRBTEST.COM; defaulting to no policy
I20260812 11:09:50.643260 24242 ts_manager.cc:194] Registered new tserver with Master: 337fa4eed860441593796983ecd2f8cd (127.17.44.194:37697)
I20260812 11:09:50.644078 24242 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.194@KRBTEST.COM'} at 127.17.44.194:57691
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
I20260812 11:09:50.663151 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:36343
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:41893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:50.795356 24770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:50.795814 24770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:50.795914 24770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:50.801662 24770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:50.801801 24770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:50.801920 24770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:09:50.809617 24770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:41893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:36343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:50.810748 24770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:50.811826 24770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:50.821815 24786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:50.821847 24782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:50.821810 24783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:50.822269 24770 server_base.cc:1034] running on GCE node
I20260812 11:09:50.822827 24770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:50.823760 24770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:50.824960 24770 hybrid_clock.cc:648] HybridClock initialized: now 1786532990824945 us; error 28 us; skew 500 ppm
Aug 12 11:09:50 dist-test-slave-drl0 krb5kdc[24168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532990, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:50.827864 24770 init.cc:377] Logged in from keytab as kudu/127.17.44.195@KRBTEST.COM (short username kudu)
I20260812 11:09:50.828658 24770 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:50.828754 24770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:50.828996 24770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:50.831246 24770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "ae61ff53018941fba1c2b499ea3115f5"
format_stamp: "Formatted at 2026-08-12 11:09:50 on dist-test-slave-drl0"
server_key: "e09a8abe566007909e74b0806990d5d5"
server_key_iv: "b15853e9ffe47fd0cb20a17c282bd25f"
server_key_version: "encryptionkey@0"
I20260812 11:09:50.831832 24770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "ae61ff53018941fba1c2b499ea3115f5"
format_stamp: "Formatted at 2026-08-12 11:09:50 on dist-test-slave-drl0"
server_key: "e09a8abe566007909e74b0806990d5d5"
server_key_iv: "b15853e9ffe47fd0cb20a17c282bd25f"
server_key_version: "encryptionkey@0"
I20260812 11:09:50.835724 24770 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 11:09:50.838239 24796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:50.839643 24770 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260812 11:09:50.839743 24770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "ae61ff53018941fba1c2b499ea3115f5"
format_stamp: "Formatted at 2026-08-12 11:09:50 on dist-test-slave-drl0"
server_key: "e09a8abe566007909e74b0806990d5d5"
server_key_iv: "b15853e9ffe47fd0cb20a17c282bd25f"
server_key_version: "encryptionkey@0"
I20260812 11:09:50.839840 24770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:50.855428 24770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:50.860092 24770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:50.860337 24770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:50.861161 24770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:50.862363 24770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:50.862448 24770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:50.862530 24770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:50.862583 24770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:50.875224 24770 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:44485
I20260812 11:09:50.875233 24985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:44485 every 8 connection(s)
I20260812 11:09:50.875980 24770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:09:50.881839 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 24770
I20260812 11:09:50.882019 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:09:50.882309 17587 external_mini_cluster.cc:1428] Setting key cab0a0947c4a2dbab45e9aaa43baffff
Aug 12 11:09:50 dist-test-slave-drl0 krb5kdc[24168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532990, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:50.892153 24305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:50.878255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:46937 (local address 127.17.44.254:36343)
0812 11:09:50.878442 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:50.878446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:50.879326 (+ 880us) server_negotiation.cc:407] Connection header received
0812 11:09:50.880688 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:50.880692 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:50.880755 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:50.880848 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:50.882127 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.882756 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.884736 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.885043 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.885177 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:50.888155 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:50.888158 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:50.888160 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:50.888183 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:50.889751 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:50.890332 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:50.890335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:50.890337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:50.890439 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:50.890911 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:50.890915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:50.890916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:50.891207 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:50.891302 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:50.891903 (+ 601us) server_negotiation.cc:299] Negotiation successful
0812 11:09:50.892024 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 11:09:50.892988 24990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:50.878421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36343 (local address 127.17.44.195:46937)
0812 11:09:50.879139 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:50.879189 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:50.880468 (+ 1279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:50.881048 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:50.881055 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:50.881418 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:50.881908 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:50.881920 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.883056 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.883062 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:50.884567 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:50.884582 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.885347 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.885353 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:50.885606 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:50.886360 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:50.886379 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:50.887972 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:50.889909 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:50.889913 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:50.889926 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:50.890180 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:50.890573 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:50.890575 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:50.890577 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:50.890741 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:50.891335 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:50.891340 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:50.891598 (+ 258us) client_negotiation.cc:769] Sending connection context
0812 11:09:50.891759 (+ 161us) client_negotiation.cc:240] Negotiation successful
0812 11:09:50.892047 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":130,"threads_started":1}
I20260812 11:09:50.894219 24986 heartbeater.cc:347] Connected to a master server at 127.17.44.254:36343
I20260812 11:09:50.894526 24986 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:50.895035 24986 heartbeater.cc:511] Master 127.17.44.254:36343 requested a full tablet report, sending...
I20260812 11:09:50.896875 24242 ts_manager.cc:194] Registered new tserver with Master: ae61ff53018941fba1c2b499ea3115f5 (127.17.44.195:44485)
I20260812 11:09:50.897663 24242 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.195@KRBTEST.COM'} at 127.17.44.195:46937
I20260812 11:09:50.897755 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:09:50.909693 24305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:50.900085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45564 (local address 127.17.44.254:36343)
0812 11:09:50.900259 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:50.900264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:50.900742 (+ 478us) server_negotiation.cc:407] Connection header received
0812 11:09:50.900821 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:50.900823 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:50.900887 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:50.900992 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:50.901825 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.902359 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.903533 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.903769 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.903878 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:50.905787 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:50.905790 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:50.905792 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:50.905814 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:50.907341 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:50.907965 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:50.907967 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:50.907969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:50.908049 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:50.908479 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:50.908484 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:50.908485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:50.908856 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:50.909010 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:50.909337 (+ 327us) server_negotiation.cc:299] Negotiation successful
0812 11:09:50.909488 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 11:09:50.913686 24242 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:09:50.917049 24242 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:09:50.934733 25009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:50.928170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:36927 (local address 127.0.0.1:45754)
0812 11:09:50.929272 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:50.929285 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:50.929374 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:50.930010 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:50.930013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:50.930031 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:09:50.930118 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:50.930124 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.931462 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.931465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:50.932802 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:50.932814 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.934151 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.934155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:50.934284 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:50.934323 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 11:09:50.934430 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 11:09:50.934544 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1022,"mutex_wait_us":2,"thread_start_us":109,"threads_started":1}
I20260812 11:09:50.934908 25011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:50.928627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45754 (local address 127.17.44.193:36927)
0812 11:09:50.929108 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:50.929116 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:50.929469 (+ 353us) server_negotiation.cc:407] Connection header received
0812 11:09:50.929550 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:50.929555 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:50.929725 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:50.929857 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:09:50.930248 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.931337 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.932965 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.933785 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.933893 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:50.934061 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:50.934619 (+ 558us) server_negotiation.cc:299] Negotiation successful
0812 11:09:50.934759 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":185,"threads_started":1}
I20260812 11:09:50.936365 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:50.927061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:37697 (local address 127.0.0.1:33360)
0812 11:09:50.928022 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:50.928158 (+ 136us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:50.928324 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:50.929545 (+ 1221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:50.929551 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:50.929574 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:09:50.929709 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:50.929716 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.931238 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.931241 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:50.933239 (+ 1998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:50.933249 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.935976 (+ 2727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.935980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:50.936068 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:50.936090 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 11:09:50.936165 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 11:09:50.936237 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":218,"thread_start_us":107,"threads_started":1}
I20260812 11:09:50.936856 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:50.927198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33360 (local address 127.17.44.194:37697)
0812 11:09:50.927565 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:50.927578 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:50.928828 (+ 1250us) server_negotiation.cc:407] Connection header received
0812 11:09:50.928923 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:50.928929 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:50.929141 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:50.929313 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:09:50.929853 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.931079 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.934832 (+ 3753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.935655 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.935765 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:50.935906 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:50.936608 (+ 702us) server_negotiation.cc:299] Negotiation successful
0812 11:09:50.936705 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":113,"threads_started":1}
I20260812 11:09:50.938365 24443 tablet_service.cc:1467] Processing CreateTablet for tablet 671d3fccb8704501b6ce18d6c5148d2a (DEFAULT_TABLE table=test-table [id=6673c449d010425983473d880c9ef916]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:50.939599 24443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 671d3fccb8704501b6ce18d6c5148d2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:50.939477 24649 tablet_service.cc:1467] Processing CreateTablet for tablet 671d3fccb8704501b6ce18d6c5148d2a (DEFAULT_TABLE table=test-table [id=6673c449d010425983473d880c9ef916]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:50.940454 24649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 671d3fccb8704501b6ce18d6c5148d2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:50.942414 25000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:50.927084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:44485 (local address 127.0.0.1:59348)
0812 11:09:50.928553 (+ 1469us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:50.928566 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:50.928642 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:50.931434 (+ 2792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:50.931439 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:50.931513 (+ 74us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:09:50.933133 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:50.933144 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.938244 (+ 5100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.938248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:50.940271 (+ 2023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:50.940294 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.941956 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.941959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:50.942052 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:50.942080 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 11:09:50.942188 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 11:09:50.942263 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1401,"mutex_wait_us":42,"thread_start_us":80,"threads_started":1}
I20260812 11:09:50.942662 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:50.927929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59348 (local address 127.17.44.195:44485)
0812 11:09:50.930316 (+ 2387us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:50.930335 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:50.930400 (+ 65us) server_negotiation.cc:407] Connection header received
0812 11:09:50.930591 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:50.930627 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:50.931094 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:50.933195 (+ 2101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:09:50.933578 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.934610 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.940878 (+ 6268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:50.941627 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:50.941740 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:50.941890 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:50.942224 (+ 334us) server_negotiation.cc:299] Negotiation successful
0812 11:09:50.942327 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2121,"thread_start_us":172,"threads_started":1}
I20260812 11:09:50.945037 24890 tablet_service.cc:1467] Processing CreateTablet for tablet 671d3fccb8704501b6ce18d6c5148d2a (DEFAULT_TABLE table=test-table [id=6673c449d010425983473d880c9ef916]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:50.946022 24890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 671d3fccb8704501b6ce18d6c5148d2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:50.948937 25013 tablet_bootstrap.cc:492] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0: Bootstrap starting.
I20260812 11:09:50.948940 25012 tablet_bootstrap.cc:492] T 671d3fccb8704501b6ce18d6c5148d2a P 337fa4eed860441593796983ecd2f8cd: Bootstrap starting.
I20260812 11:09:50.951363 25013 tablet_bootstrap.cc:654] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:50.951363 25012 tablet_bootstrap.cc:654] T 671d3fccb8704501b6ce18d6c5148d2a P 337fa4eed860441593796983ecd2f8cd: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:50.952344 25013 log.cc:824] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:50.952616 25012 log.cc:824] T 671d3fccb8704501b6ce18d6c5148d2a P 337fa4eed860441593796983ecd2f8cd: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:50.952971 25015 tablet_bootstrap.cc:492] T 671d3fccb8704501b6ce18d6c5148d2a P ae61ff53018941fba1c2b499ea3115f5: Bootstrap starting.
I20260812 11:09:50.954320 25013 tablet_bootstrap.cc:492] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0: No bootstrap required, opened a new log
I20260812 11:09:50.954558 25013 ts_tablet_manager.cc:1397] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:09:50.954700 25012 tablet_bootstrap.cc:492] T 671d3fccb8704501b6ce18d6c5148d2a P 337fa4eed860441593796983ecd2f8cd: No bootstrap required, opened a new log
I20260812 11:09:50.954886 25012 ts_tablet_manager.cc:1397] T 671d3fccb8704501b6ce18d6c5148d2a P 337fa4eed860441593796983ecd2f8cd: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 11:09:50.955502 25015 tablet_bootstrap.cc:654] T 671d3fccb8704501b6ce18d6c5148d2a P ae61ff53018941fba1c2b499ea3115f5: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:50.956849 25015 log.cc:824] T 671d3fccb8704501b6ce18d6c5148d2a P ae61ff53018941fba1c2b499ea3115f5: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:50.958436 25013 raft_consensus.cc:348] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337fa4eed860441593796983ecd2f8cd" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 37697 } } peers { permanent_uuid: "ae61ff53018941fba1c2b499ea3115f5" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 44485 } } peers { permanent_uuid: "6a061687201b47acbc76fb41dfa192c0" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36927 } }
I20260812 11:09:50.958678 25013 raft_consensus.cc:374] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:50.958861 25013 raft_consensus.cc:729] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a061687201b47acbc76fb41dfa192c0, State: Initialized, Role: FOLLOWER
I20260812 11:09:50.958910 25015 tablet_bootstrap.cc:492] T 671d3fccb8704501b6ce18d6c5148d2a P ae61ff53018941fba1c2b499ea3115f5: No bootstrap required, opened a new log
I20260812 11:09:50.958856 25012 raft_consensus.cc:348] T 671d3fccb8704501b6ce18d6c5148d2a P 337fa4eed860441593796983ecd2f8cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337fa4eed860441593796983ecd2f8cd" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 37697 } } peers { permanent_uuid: "ae61ff53018941fba1c2b499ea3115f5" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 44485 } } peers { permanent_uuid: "6a061687201b47acbc76fb41dfa192c0" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36927 } }
I20260812 11:09:50.959149 25012 raft_consensus.cc:374] T 671d3fccb8704501b6ce18d6c5148d2a P 337fa4eed860441593796983ecd2f8cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:50.959201 25012 raft_consensus.cc:729] T 671d3fccb8704501b6ce18d6c5148d2a P 337fa4eed860441593796983ecd2f8cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 337fa4eed860441593796983ecd2f8cd, State: Initialized, Role: FOLLOWER
I20260812 11:09:50.959474 25013 consensus_queue.cc:260] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337fa4eed860441593796983ecd2f8cd" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 37697 } } peers { permanent_uuid: "ae61ff53018941fba1c2b499ea3115f5" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 44485 } } peers { permanent_uuid: "6a061687201b47acbc76fb41dfa192c0" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36927 } }
I20260812 11:09:50.959637 25015 ts_tablet_manager.cc:1397] T 671d3fccb8704501b6ce18d6c5148d2a P ae61ff53018941fba1c2b499ea3115f5: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.002s
I20260812 11:09:50.959621 25012 consensus_queue.cc:260] T 671d3fccb8704501b6ce18d6c5148d2a P 337fa4eed860441593796983ecd2f8cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337fa4eed860441593796983ecd2f8cd" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 37697 } } peers { permanent_uuid: "ae61ff53018941fba1c2b499ea3115f5" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 44485 } } peers { permanent_uuid: "6a061687201b47acbc76fb41dfa192c0" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36927 } }
I20260812 11:09:50.960337 24759 heartbeater.cc:503] Master 127.17.44.254:36343 was elected leader, sending a full tablet report...
I20260812 11:09:50.960598 25013 ts_tablet_manager.cc:1428] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:09:50.960964 24548 heartbeater.cc:503] Master 127.17.44.254:36343 was elected leader, sending a full tablet report...
I20260812 11:09:50.961515 25012 ts_tablet_manager.cc:1428] T 671d3fccb8704501b6ce18d6c5148d2a P 337fa4eed860441593796983ecd2f8cd: Time spent starting tablet: real 0.007s user 0.003s sys 0.002s
I20260812 11:09:50.965200 25015 raft_consensus.cc:348] T 671d3fccb8704501b6ce18d6c5148d2a P ae61ff53018941fba1c2b499ea3115f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337fa4eed860441593796983ecd2f8cd" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 37697 } } peers { permanent_uuid: "ae61ff53018941fba1c2b499ea3115f5" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 44485 } } peers { permanent_uuid: "6a061687201b47acbc76fb41dfa192c0" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36927 } }
I20260812 11:09:50.965523 25015 raft_consensus.cc:374] T 671d3fccb8704501b6ce18d6c5148d2a P ae61ff53018941fba1c2b499ea3115f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:50.965595 25015 raft_consensus.cc:729] T 671d3fccb8704501b6ce18d6c5148d2a P ae61ff53018941fba1c2b499ea3115f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae61ff53018941fba1c2b499ea3115f5, State: Initialized, Role: FOLLOWER
I20260812 11:09:50.965989 25015 consensus_queue.cc:260] T 671d3fccb8704501b6ce18d6c5148d2a P ae61ff53018941fba1c2b499ea3115f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337fa4eed860441593796983ecd2f8cd" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 37697 } } peers { permanent_uuid: "ae61ff53018941fba1c2b499ea3115f5" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 44485 } } peers { permanent_uuid: "6a061687201b47acbc76fb41dfa192c0" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36927 } }
I20260812 11:09:50.966638 24986 heartbeater.cc:503] Master 127.17.44.254:36343 was elected leader, sending a full tablet report...
I20260812 11:09:50.966846 25015 ts_tablet_manager.cc:1428] T 671d3fccb8704501b6ce18d6c5148d2a P ae61ff53018941fba1c2b499ea3115f5: Time spent starting tablet: real 0.007s user 0.001s sys 0.004s
I20260812 11:09:51.011919 25023 raft_consensus.cc:482] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:09:51.012182 25023 raft_consensus.cc:504] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337fa4eed860441593796983ecd2f8cd" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 37697 } } peers { permanent_uuid: "ae61ff53018941fba1c2b499ea3115f5" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 44485 } } peers { permanent_uuid: "6a061687201b47acbc76fb41dfa192c0" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36927 } }
I20260812 11:09:51.013438 25023 leader_election.cc:288] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 337fa4eed860441593796983ecd2f8cd (127.17.44.194:37697), ae61ff53018941fba1c2b499ea3115f5 (127.17.44.195:44485)
I20260812 11:09:51.020937 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:51.013847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:49007 (local address 127.17.44.195:44485)
0812 11:09:51.013993 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:51.013998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:51.014577 (+ 579us) server_negotiation.cc:407] Connection header received
0812 11:09:51.014721 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:51.014725 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:51.014779 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:51.014874 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:09:51.015643 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:51.016385 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:51.018749 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:51.019607 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:51.019802 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:51.019877 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:51.020655 (+ 778us) server_negotiation.cc:299] Negotiation successful
0812 11:09:51.020769 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 11:09:51.020936 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:51.013911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:49915 (local address 127.17.44.194:37697)
0812 11:09:51.014049 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:51.014054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:51.014580 (+ 526us) server_negotiation.cc:407] Connection header received
0812 11:09:51.014721 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:51.014725 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:51.014779 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:51.014874 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:09:51.015492 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:51.016318 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:51.018749 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:51.019623 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:51.019803 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:51.019877 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:51.020655 (+ 778us) server_negotiation.cc:299] Negotiation successful
0812 11:09:51.020769 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 11:09:51.021011 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:51.013761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:44485 (local address 127.17.44.193:49007)
0812 11:09:51.014317 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:51.014334 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:51.014594 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:51.015364 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:51.015366 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:51.015384 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:09:51.015498 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:51.015504 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:51.016573 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:51.016575 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:51.018529 (+ 1954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:51.018555 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:51.019853 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:51.019857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:51.020077 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:51.020458 (+ 381us) client_negotiation.cc:769] Sending connection context
0812 11:09:51.020572 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 11:09:51.020739 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":429,"spinlock_wait_cycles":38528,"thread_start_us":141,"threads_started":1}
I20260812 11:09:51.021011 25030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:51.013821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:37697 (local address 127.17.44.193:49915)
0812 11:09:51.014383 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:51.014416 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:51.014594 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:51.015057 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:51.015061 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:51.015171 (+ 110us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:09:51.015346 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:51.015354 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:51.016556 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:51.016562 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:51.018529 (+ 1967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:51.018549 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:51.019852 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:51.019857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:51.020102 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:51.020464 (+ 362us) client_negotiation.cc:769] Sending connection context
0812 11:09:51.020589 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 11:09:51.020739 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":385,"spinlock_wait_cycles":11904,"thread_start_us":142,"threads_started":1}
I20260812 11:09:51.021891 24917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "671d3fccb8704501b6ce18d6c5148d2a" candidate_uuid: "6a061687201b47acbc76fb41dfa192c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae61ff53018941fba1c2b499ea3115f5" is_pre_election: true
I20260812 11:09:51.022297 24917 raft_consensus.cc:2454] T 671d3fccb8704501b6ce18d6c5148d2a P ae61ff53018941fba1c2b499ea3115f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a061687201b47acbc76fb41dfa192c0 in term 0.
I20260812 11:09:51.022361 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "671d3fccb8704501b6ce18d6c5148d2a" candidate_uuid: "6a061687201b47acbc76fb41dfa192c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337fa4eed860441593796983ecd2f8cd" is_pre_election: true
I20260812 11:09:51.022625 24694 raft_consensus.cc:2454] T 671d3fccb8704501b6ce18d6c5148d2a P 337fa4eed860441593796983ecd2f8cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a061687201b47acbc76fb41dfa192c0 in term 0.
I20260812 11:09:51.022960 24364 leader_election.cc:302] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a061687201b47acbc76fb41dfa192c0, ae61ff53018941fba1c2b499ea3115f5; no voters:
I20260812 11:09:51.023267 25023 raft_consensus.cc:2781] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:09:51.023345 25023 raft_consensus.cc:482] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:09:51.023422 25023 raft_consensus.cc:3037] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:51.024542 25023 raft_consensus.cc:504] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337fa4eed860441593796983ecd2f8cd" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 37697 } } peers { permanent_uuid: "ae61ff53018941fba1c2b499ea3115f5" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 44485 } } peers { permanent_uuid: "6a061687201b47acbc76fb41dfa192c0" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36927 } }
I20260812 11:09:51.024966 25023 leader_election.cc:288] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0 [CANDIDATE]: Term 1 election: Requested vote from peers 337fa4eed860441593796983ecd2f8cd (127.17.44.194:37697), ae61ff53018941fba1c2b499ea3115f5 (127.17.44.195:44485)
I20260812 11:09:51.025434 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "671d3fccb8704501b6ce18d6c5148d2a" candidate_uuid: "6a061687201b47acbc76fb41dfa192c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "337fa4eed860441593796983ecd2f8cd"
I20260812 11:09:51.025578 24694 raft_consensus.cc:3037] T 671d3fccb8704501b6ce18d6c5148d2a P 337fa4eed860441593796983ecd2f8cd [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:51.025663 24917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "671d3fccb8704501b6ce18d6c5148d2a" candidate_uuid: "6a061687201b47acbc76fb41dfa192c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae61ff53018941fba1c2b499ea3115f5"
I20260812 11:09:51.025923 24917 raft_consensus.cc:3037] T 671d3fccb8704501b6ce18d6c5148d2a P ae61ff53018941fba1c2b499ea3115f5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:51.026880 24694 raft_consensus.cc:2454] T 671d3fccb8704501b6ce18d6c5148d2a P 337fa4eed860441593796983ecd2f8cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a061687201b47acbc76fb41dfa192c0 in term 1.
I20260812 11:09:51.027251 24367 leader_election.cc:302] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 337fa4eed860441593796983ecd2f8cd, 6a061687201b47acbc76fb41dfa192c0; no voters:
I20260812 11:09:51.027365 24917 raft_consensus.cc:2454] T 671d3fccb8704501b6ce18d6c5148d2a P ae61ff53018941fba1c2b499ea3115f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a061687201b47acbc76fb41dfa192c0 in term 1.
I20260812 11:09:51.027439 25023 raft_consensus.cc:2781] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:51.027698 25023 raft_consensus.cc:686] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0 [term 1 LEADER]: Becoming Leader. State: Replica: 6a061687201b47acbc76fb41dfa192c0, State: Running, Role: LEADER
I20260812 11:09:51.028095 25023 consensus_queue.cc:237] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337fa4eed860441593796983ecd2f8cd" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 37697 } } peers { permanent_uuid: "ae61ff53018941fba1c2b499ea3115f5" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 44485 } } peers { permanent_uuid: "6a061687201b47acbc76fb41dfa192c0" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36927 } }
I20260812 11:09:51.031584 24242 catalog_manager.cc:5495] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a061687201b47acbc76fb41dfa192c0 (127.17.44.193). New cstate: current_term: 1 leader_uuid: "6a061687201b47acbc76fb41dfa192c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "337fa4eed860441593796983ecd2f8cd" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 37697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae61ff53018941fba1c2b499ea3115f5" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 44485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a061687201b47acbc76fb41dfa192c0" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36927 } health_report { overall_health: HEALTHY } } }
I20260812 11:09:51.061717 25011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:51.057773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45760 (local address 127.17.44.193:36927)
0812 11:09:51.057959 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:51.057966 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:51.057982 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:09:51.058202 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:51.058205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:51.058264 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:51.058387 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:09:51.058654 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:51.059322 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:51.060178 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:51.060402 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:51.060517 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:51.060845 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:09:51.061297 (+ 452us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:09:51.061433 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:51.061496 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 11:09:51.061554 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 11:09:51.069752 24917 raft_consensus.cc:1261] T 671d3fccb8704501b6ce18d6c5148d2a P ae61ff53018941fba1c2b499ea3115f5 [term 1 FOLLOWER]: Refusing update from remote peer 6a061687201b47acbc76fb41dfa192c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:09:51.069626 24694 raft_consensus.cc:1261] T 671d3fccb8704501b6ce18d6c5148d2a P 337fa4eed860441593796983ecd2f8cd [term 1 FOLLOWER]: Refusing update from remote peer 6a061687201b47acbc76fb41dfa192c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:09:51.070587 25032 consensus_queue.cc:1035] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "337fa4eed860441593796983ecd2f8cd" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 37697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:09:51.070995 25023 consensus_queue.cc:1035] T 671d3fccb8704501b6ce18d6c5148d2a P 6a061687201b47acbc76fb41dfa192c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae61ff53018941fba1c2b499ea3115f5" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 44485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:09:51.079190 25035 mvcc.cc:205] Tried to move back new op lower bound from 7317639131411206144 to 7317639131253714944. Current Snapshot: MvccSnapshot[applied={T|T < 7317639131411206144}]
I20260812 11:09:51.080114 25039 mvcc.cc:205] Tried to move back new op lower bound from 7317639131411206144 to 7317639131253714944. Current Snapshot: MvccSnapshot[applied={T|T < 7317639131411206144}]
I20260812 11:09:51.082967 25037 mvcc.cc:205] Tried to move back new op lower bound from 7317639131411206144 to 7317639131253714944. Current Snapshot: MvccSnapshot[applied={T|T < 7317639131411206144}]
I20260812 11:09:51.089643 24238 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:09:51.089833 24238 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:09:51.092545 24238 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 85ef49b0a8904f9eb7c219dc1c64a445: Sending DeleteTablet for 3 replicas of tablet 671d3fccb8704501b6ce18d6c5148d2a
I20260812 11:09:51.093343 24649 tablet_service.cc:1514] Processing DeleteTablet for tablet 671d3fccb8704501b6ce18d6c5148d2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:09:51 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:33360
I20260812 11:09:51.093377 24890 tablet_service.cc:1514] Processing DeleteTablet for tablet 671d3fccb8704501b6ce18d6c5148d2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:09:51 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:59348
I20260812 11:09:51.093699 24443 tablet_service.cc:1514] Processing DeleteTablet for tablet 671d3fccb8704501b6ce18d6c5148d2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:09:51 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:45754
I20260812 11:09:51.094226 25051 tablet_replica.cc:330] T 671d3fccb8704501b6ce18d6c5148d2a P ae61ff53018941fba1c2b499ea3115f5: stopping tablet replica
I20260812 11:09:51.094581 25051 raft_consensus.cc:2229] T 671d3fccb8704501b6ce18d6c5148d2a P ae61ff53018941fba1c2b499ea3115f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:09:51.094727 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 24338
I20260812 11:09:51.100504 25051 raft_consensus.cc:2258] T 671d3fccb8704501b6ce18d6c5148d2a P ae61ff53018941fba1c2b499ea3115f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:09:51.103595 25051 ts_tablet_manager.cc:1905] T 671d3fccb8704501b6ce18d6c5148d2a P ae61ff53018941fba1c2b499ea3115f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:09:51.107000 25051 ts_tablet_manager.cc:1918] T 671d3fccb8704501b6ce18d6c5148d2a P ae61ff53018941fba1c2b499ea3115f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:09:51.107141 25051 log.cc:1196] T 671d3fccb8704501b6ce18d6c5148d2a P ae61ff53018941fba1c2b499ea3115f5: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/671d3fccb8704501b6ce18d6c5148d2a
I20260812 11:09:51.107570 25051 ts_tablet_manager.cc:1939] T 671d3fccb8704501b6ce18d6c5148d2a P ae61ff53018941fba1c2b499ea3115f5: Deleting consensus metadata
I20260812 11:09:51.108763 24214 catalog_manager.cc:4841] TS ae61ff53018941fba1c2b499ea3115f5 (127.17.44.195:44485): tablet 671d3fccb8704501b6ce18d6c5148d2a (table test-table [id=6673c449d010425983473d880c9ef916]) successfully deleted
I20260812 11:09:51.113317 25050 tablet_replica.cc:330] T 671d3fccb8704501b6ce18d6c5148d2a P 337fa4eed860441593796983ecd2f8cd: stopping tablet replica
I20260812 11:09:51.113722 25050 raft_consensus.cc:2229] T 671d3fccb8704501b6ce18d6c5148d2a P 337fa4eed860441593796983ecd2f8cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:09:51.114075 25050 raft_consensus.cc:2258] T 671d3fccb8704501b6ce18d6c5148d2a P 337fa4eed860441593796983ecd2f8cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:09:51.115697 25050 ts_tablet_manager.cc:1905] T 671d3fccb8704501b6ce18d6c5148d2a P 337fa4eed860441593796983ecd2f8cd: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 11:09:51.121124 24218 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.44.193:36927 (error 108)
I20260812 11:09:51.121302 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 24554
I20260812 11:09:51.123842 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:51.123478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:36927 (local address 127.0.0.1:45764)
0812 11:09:51.123650 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:51.123728 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.193:36927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 11:09:51.124011 24218 catalog_manager.cc:4584] TS 6a061687201b47acbc76fb41dfa192c0 (127.17.44.193:36927): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 671d3fccb8704501b6ce18d6c5148d2a: Network error: Client connection negotiation failed: client connection to 127.17.44.193:36927: connect: Connection refused (error 111)
I20260812 11:09:51.137648 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 24770
I20260812 11:09:51.145161 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:51.137847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:37697 (local address 127.0.0.1:33366)
0812 11:09:51.144946 (+ 7099us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:51.145034 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.194:37697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6969}
I20260812 11:09:51.151791 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 24193
2026-08-12T11:09:51Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (3857 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T11:09:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:09:51Z Disabled control of system clock
I20260812 11:09:51.189843 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:43371
--webserver_interface=127.17.44.254
--webserver_port=0
--builtin_ntp_servers=127.17.44.212:36237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:43371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 11:09:51.309805 25065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:51.310063 25065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:51.310099 25065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:51.314128 25065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:09:51.314189 25065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:51.314205 25065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:51.314222 25065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:09:51.314237 25065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:09:51.319187 25065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:36237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:43371
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:43371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:51.320175 25065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:51.321532 25065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:51.331044 25077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:51.331032 25076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:51.331028 25079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:51.331559 25065 server_base.cc:1034] running on GCE node
I20260812 11:09:51.332190 25065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:51.333257 25065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:51.334498 25065 hybrid_clock.cc:648] HybridClock initialized: now 1786532991334462 us; error 41 us; skew 500 ppm
I20260812 11:09:51.336525 25065 webserver.cc:460] Webserver started at http://127.17.44.254:46739/ using document root <none> and password file <none>
I20260812 11:09:51.337111 25065 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:51.337212 25065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:51.337457 25065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:51.339553 25065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "58caea7c93504c17a713de996f7143c8"
format_stamp: "Formatted at 2026-08-12 11:09:51 on dist-test-slave-drl0"
server_key: "9e45d5ccff8a83a12d18a275ffc47e44"
server_key_iv: "bf729b4598de620bd9eed476f38141f6"
server_key_version: "encryptionkey@0"
I20260812 11:09:51.340116 25065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "58caea7c93504c17a713de996f7143c8"
format_stamp: "Formatted at 2026-08-12 11:09:51 on dist-test-slave-drl0"
server_key: "9e45d5ccff8a83a12d18a275ffc47e44"
server_key_iv: "bf729b4598de620bd9eed476f38141f6"
server_key_version: "encryptionkey@0"
I20260812 11:09:51.344260 25065 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 11:09:51.347240 25087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:51.348703 25065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 11:09:51.348865 25065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "58caea7c93504c17a713de996f7143c8"
format_stamp: "Formatted at 2026-08-12 11:09:51 on dist-test-slave-drl0"
server_key: "9e45d5ccff8a83a12d18a275ffc47e44"
server_key_iv: "bf729b4598de620bd9eed476f38141f6"
server_key_version: "encryptionkey@0"
I20260812 11:09:51.348985 25065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:51.354660 25065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:51.355877 25065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:51.356148 25065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:51.364714 25065 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:43371
I20260812 11:09:51.364738 25170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:43371 every 8 connection(s)
I20260812 11:09:51.365685 25065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:09:51.366088 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 25065
I20260812 11:09:51.366235 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:09:51.366552 17587 external_mini_cluster.cc:1428] Setting key b46fffe6d5a0a98b0732885fd5ee546e
I20260812 11:09:51.368714 25171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:51.372483 25175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:51.368006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59406 (local address 127.17.44.254:43371)
0812 11:09:51.369073 (+ 1067us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:51.369089 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:51.369123 (+ 34us) server_negotiation.cc:407] Connection header received
0812 11:09:51.369752 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:51.369767 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:51.370098 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:51.370384 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:51.371224 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:51.371228 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:51.371239 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:09:51.371249 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:51.371302 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:09:51.371328 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:51.371438 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:51.371548 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 11:09:51.371794 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":578,"threads_started":1}
I20260812 11:09:51.374897 25171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8: Bootstrap starting.
I20260812 11:09:51.376981 25171 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:51.377712 25171 log.cc:824] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:51.379652 25171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8: No bootstrap required, opened a new log
I20260812 11:09:51.383529 25171 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58caea7c93504c17a713de996f7143c8" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43371 } }
I20260812 11:09:51.383761 25171 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:51.383841 25171 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58caea7c93504c17a713de996f7143c8, State: Initialized, Role: FOLLOWER
I20260812 11:09:51.384290 25171 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58caea7c93504c17a713de996f7143c8" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43371 } }
I20260812 11:09:51.384444 25171 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:09:51.384533 25171 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:09:51.384636 25171 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:51.385493 25171 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58caea7c93504c17a713de996f7143c8" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43371 } }
I20260812 11:09:51.385826 25171 leader_election.cc:302] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58caea7c93504c17a713de996f7143c8; no voters:
I20260812 11:09:51.386132 25171 leader_election.cc:288] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:09:51.386581 25179 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:51.386816 25179 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8 [term 1 LEADER]: Becoming Leader. State: Replica: 58caea7c93504c17a713de996f7143c8, State: Running, Role: LEADER
I20260812 11:09:51.387162 25179 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58caea7c93504c17a713de996f7143c8" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43371 } }
I20260812 11:09:51.387501 25171 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:09:51.388725 25181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58caea7c93504c17a713de996f7143c8. Latest consensus state: current_term: 1 leader_uuid: "58caea7c93504c17a713de996f7143c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58caea7c93504c17a713de996f7143c8" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43371 } } }
I20260812 11:09:51.388873 25181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8 [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:51.389204 25180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58caea7c93504c17a713de996f7143c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58caea7c93504c17a713de996f7143c8" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43371 } } }
I20260812 11:09:51.389317 25180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8 [sys.catalog]: This master's current role is: LEADER
W20260812 11:09:51.392381 25203 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:09:51.392482 25203 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:09:51.392575 25191 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:09:51.393471 25191 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:09:51.398856 25191 catalog_manager.cc:1347] Generated new cluster ID: 8719929199e2475bb29d526981659973
I20260812 11:09:51.398921 25191 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:09:51.412680 25191 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:09:51.413017 25191 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:09:51.419618 25191 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8: Generated new TSK 0
I20260812 11:09:51.420341 25191 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:09:51.427542 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:43371
--builtin_ntp_servers=127.17.44.212:36237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 11:09:51.556108 25211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:51.556524 25211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:51.556615 25211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:51.561873 25211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:51.562007 25211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:51.562125 25211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:09:51.568581 25211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:36237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:43371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:51.570497 25211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:51.572065 25211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:51.584686 25219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:51.584674 25218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:51.584674 25223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:51.585314 25211 server_base.cc:1034] running on GCE node
I20260812 11:09:51.585755 25211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:51.586483 25211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:51.587718 25211 hybrid_clock.cc:648] HybridClock initialized: now 1786532991587642 us; error 45 us; skew 500 ppm
I20260812 11:09:51.590690 25211 webserver.cc:460] Webserver started at http://127.17.44.193:40663/ using document root <none> and password file <none>
I20260812 11:09:51.591380 25211 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:51.591450 25211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:51.591683 25211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:51.593799 25211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "2b851795a3e940d6b1b2aa90e5036172"
format_stamp: "Formatted at 2026-08-12 11:09:51 on dist-test-slave-drl0"
server_key: "e98224b812aea0e8ca58faa7db8a1cba"
server_key_iv: "e0176e318891acbe1db2de12ff95f8dd"
server_key_version: "encryptionkey@0"
I20260812 11:09:51.594391 25211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "2b851795a3e940d6b1b2aa90e5036172"
format_stamp: "Formatted at 2026-08-12 11:09:51 on dist-test-slave-drl0"
server_key: "e98224b812aea0e8ca58faa7db8a1cba"
server_key_iv: "e0176e318891acbe1db2de12ff95f8dd"
server_key_version: "encryptionkey@0"
I20260812 11:09:51.598212 25211 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 11:09:51.601325 25231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:51.602649 25211 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:09:51.602751 25211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "2b851795a3e940d6b1b2aa90e5036172"
format_stamp: "Formatted at 2026-08-12 11:09:51 on dist-test-slave-drl0"
server_key: "e98224b812aea0e8ca58faa7db8a1cba"
server_key_iv: "e0176e318891acbe1db2de12ff95f8dd"
server_key_version: "encryptionkey@0"
I20260812 11:09:51.603040 25211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:51.608649 25211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:51.609292 25211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:51.609483 25211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:51.610041 25211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:51.611349 25211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:51.611411 25211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:51.611467 25211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:51.611500 25211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:51.623026 25211 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:40309
I20260812 11:09:51.623240 25406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:40309 every 8 connection(s)
I20260812 11:09:51.624006 25211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:09:51.626256 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 25211
I20260812 11:09:51.626451 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:09:51.626801 17587 external_mini_cluster.cc:1428] Setting key c3a80e9238848ac2e072d08df1a03690
I20260812 11:09:51.629570 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:43371
--builtin_ntp_servers=127.17.44.212:36237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 11:09:51.632915 25175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:51.626438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:33639 (local address 127.17.44.254:43371)
0812 11:09:51.626620 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:51.626626 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:51.627965 (+ 1339us) server_negotiation.cc:407] Connection header received
0812 11:09:51.629980 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:51.629986 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:51.630036 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:51.630116 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:51.632306 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:51.632310 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:51.632313 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:09:51.632325 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:51.632350 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:09:51.632365 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:51.632562 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:51.632716 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 11:09:51.632763 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 11:09:51.633839 25411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:51.626738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:43371 (local address 127.17.44.193:33639)
0812 11:09:51.627743 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:51.627796 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:51.629589 (+ 1793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:51.631126 (+ 1537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:51.631138 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:51.631252 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:51.631980 (+ 728us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:09:51.631985 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:51.632049 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:09:51.632154 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:51.632544 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:51.632555 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:51.632561 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 11:09:51.632755 (+ 194us) client_negotiation.cc:240] Negotiation successful
0812 11:09:51.632993 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":176,"threads_started":1}
I20260812 11:09:51.635143 25407 heartbeater.cc:347] Connected to a master server at 127.17.44.254:43371
I20260812 11:09:51.635383 25407 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:51.635823 25407 heartbeater.cc:511] Master 127.17.44.254:43371 requested a full tablet report, sending...
I20260812 11:09:51.637660 25113 ts_manager.cc:194] Registered new tserver with Master: 2b851795a3e940d6b1b2aa90e5036172 (127.17.44.193:40309)
W20260812 11:09:51.761003 25412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:51.761256 25412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:51.761296 25412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:51.765345 25412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:51.765403 25412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:51.765457 25412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:09:51.770363 25412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:36237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:43371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:51.771440 25412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:51.772830 25412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:51.783686 25427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:51.783692 25421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:51.783712 25422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:51.784548 25412 server_base.cc:1034] running on GCE node
I20260812 11:09:51.784953 25412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:51.785574 25412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:51.786791 25412 hybrid_clock.cc:648] HybridClock initialized: now 1786532991786773 us; error 31 us; skew 500 ppm
I20260812 11:09:51.789453 25412 webserver.cc:460] Webserver started at http://127.17.44.194:35493/ using document root <none> and password file <none>
I20260812 11:09:51.790047 25412 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:51.790143 25412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:51.790372 25412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:51.792595 25412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "69aaab9af579491e832f4822dddf45d9"
format_stamp: "Formatted at 2026-08-12 11:09:51 on dist-test-slave-drl0"
server_key: "dd5b855db6efcb961cd134b55ac5404c"
server_key_iv: "ca73c7c000cd96be43cf4fe3b6471bfe"
server_key_version: "encryptionkey@0"
I20260812 11:09:51.793179 25412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "69aaab9af579491e832f4822dddf45d9"
format_stamp: "Formatted at 2026-08-12 11:09:51 on dist-test-slave-drl0"
server_key: "dd5b855db6efcb961cd134b55ac5404c"
server_key_iv: "ca73c7c000cd96be43cf4fe3b6471bfe"
server_key_version: "encryptionkey@0"
I20260812 11:09:51.797097 25412 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260812 11:09:51.800505 25437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:51.802101 25412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 11:09:51.802248 25412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "69aaab9af579491e832f4822dddf45d9"
format_stamp: "Formatted at 2026-08-12 11:09:51 on dist-test-slave-drl0"
server_key: "dd5b855db6efcb961cd134b55ac5404c"
server_key_iv: "ca73c7c000cd96be43cf4fe3b6471bfe"
server_key_version: "encryptionkey@0"
I20260812 11:09:51.802371 25412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:51.807030 25412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:51.807633 25412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:51.807818 25412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:51.808437 25412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:51.810580 25412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:51.810678 25412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:51.810734 25412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:51.810812 25412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:51.822230 25412 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:45821
I20260812 11:09:51.822249 25611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:45821 every 8 connection(s)
I20260812 11:09:51.823334 25412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:09:51.829617 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 25412
I20260812 11:09:51.829775 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:09:51.830171 17587 external_mini_cluster.cc:1428] Setting key f771af779cc5e1bc36fb1e9f70ef6a66
I20260812 11:09:51.830998 25175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:51.825455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:44861 (local address 127.17.44.254:43371)
0812 11:09:51.825617 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:51.825622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:51.827146 (+ 1524us) server_negotiation.cc:407] Connection header received
0812 11:09:51.828417 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:51.828420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:51.828462 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:51.828558 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:51.830433 (+ 1875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:51.830437 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:51.830440 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:09:51.830453 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:51.830476 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:09:51.830487 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:51.830552 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:51.830790 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 11:09:51.830838 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 11:09:51.832021 25616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:51.825859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:43371 (local address 127.17.44.194:44861)
0812 11:09:51.826902 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:51.827023 (+ 121us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:51.828211 (+ 1188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:51.828905 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:51.828916 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:51.829159 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:51.830080 (+ 921us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:09:51.830083 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:51.830139 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:09:51.830240 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:51.830635 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:51.830652 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:51.830657 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 11:09:51.830839 (+ 182us) client_negotiation.cc:240] Negotiation successful
0812 11:09:51.831105 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":150,"threads_started":1}
I20260812 11:09:51.832418 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:43371
--builtin_ntp_servers=127.17.44.212:36237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 11:09:51.834009 25614 heartbeater.cc:347] Connected to a master server at 127.17.44.254:43371
I20260812 11:09:51.834466 25614 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:51.835286 25614 heartbeater.cc:511] Master 127.17.44.254:43371 requested a full tablet report, sending...
I20260812 11:09:51.836800 25113 ts_manager.cc:194] Registered new tserver with Master: 69aaab9af579491e832f4822dddf45d9 (127.17.44.194:45821)
W20260812 11:09:51.964223 25617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:51.964468 25617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:51.964509 25617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:51.968484 25617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:51.968531 25617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:51.968587 25617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:09:51.974006 25617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:36237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:43371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:51.975087 25617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:51.976511 25617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:51.987684 25630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:51.987712 25626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:51.987690 25628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:51.988142 25617 server_base.cc:1034] running on GCE node
I20260812 11:09:51.988564 25617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:51.989394 25617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:51.990720 25617 hybrid_clock.cc:648] HybridClock initialized: now 1786532991990654 us; error 47 us; skew 500 ppm
I20260812 11:09:51.993731 25617 webserver.cc:460] Webserver started at http://127.17.44.195:37553/ using document root <none> and password file <none>
I20260812 11:09:51.994566 25617 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:51.994693 25617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:51.995150 25617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:51.997318 25617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "1e0113b252554fad9b90ddfe84ee0c03"
format_stamp: "Formatted at 2026-08-12 11:09:51 on dist-test-slave-drl0"
server_key: "50dd3ace5e5f41d9950221c2c4dc2715"
server_key_iv: "c5d686fc1f78be0a5465e85c96014514"
server_key_version: "encryptionkey@0"
I20260812 11:09:51.997884 25617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "1e0113b252554fad9b90ddfe84ee0c03"
format_stamp: "Formatted at 2026-08-12 11:09:51 on dist-test-slave-drl0"
server_key: "50dd3ace5e5f41d9950221c2c4dc2715"
server_key_iv: "c5d686fc1f78be0a5465e85c96014514"
server_key_version: "encryptionkey@0"
I20260812 11:09:52.001870 25617 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260812 11:09:52.004526 25640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:52.005980 25617 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 11:09:52.006135 25617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "1e0113b252554fad9b90ddfe84ee0c03"
format_stamp: "Formatted at 2026-08-12 11:09:51 on dist-test-slave-drl0"
server_key: "50dd3ace5e5f41d9950221c2c4dc2715"
server_key_iv: "c5d686fc1f78be0a5465e85c96014514"
server_key_version: "encryptionkey@0"
I20260812 11:09:52.006258 25617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:52.011288 25617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:52.011915 25617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:52.012105 25617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:52.013197 25617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:52.014221 25617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:52.014295 25617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:52.014362 25617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:52.014405 25617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:52.024868 25617 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:42397
I20260812 11:09:52.024978 25833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:42397 every 8 connection(s)
I20260812 11:09:52.025800 25617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:09:52.032135 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 25617
I20260812 11:09:52.032261 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:09:52.032603 17587 external_mini_cluster.cc:1428] Setting key 7af710e474756bf3bf280be8eef60d3f
I20260812 11:09:52.032867 25175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:52.028009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:39259 (local address 127.17.44.254:43371)
0812 11:09:52.028150 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:52.028154 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:52.029384 (+ 1230us) server_negotiation.cc:407] Connection header received
0812 11:09:52.030718 (+ 1334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:52.030721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:52.030766 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:52.031016 (+ 250us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:52.032295 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:52.032301 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:52.032305 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:09:52.032321 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:52.032352 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:09:52.032361 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:52.032459 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:52.032680 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 11:09:52.032736 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:09:52.033887 25838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:52.028272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:43371 (local address 127.17.44.195:39259)
0812 11:09:52.029216 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:52.029260 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:52.030509 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:52.031171 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:52.031178 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:52.031262 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:52.031921 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:09:52.031925 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:52.031985 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:09:52.032073 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:52.032462 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:52.032472 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:52.032479 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 11:09:52.032666 (+ 187us) client_negotiation.cc:240] Negotiation successful
0812 11:09:52.032918 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":124,"threads_started":1}
I20260812 11:09:52.035305 25835 heartbeater.cc:347] Connected to a master server at 127.17.44.254:43371
I20260812 11:09:52.035530 25835 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:52.035962 25835 heartbeater.cc:511] Master 127.17.44.254:43371 requested a full tablet report, sending...
I20260812 11:09:52.037217 25116 ts_manager.cc:194] Registered new tserver with Master: 1e0113b252554fad9b90ddfe84ee0c03 (127.17.44.195:42397)
I20260812 11:09:52.047839 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:09:52.052058 25175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:52.050033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59408 (local address 127.17.44.254:43371)
0812 11:09:52.050161 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:52.050165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:52.050481 (+ 316us) server_negotiation.cc:407] Connection header received
0812 11:09:52.050644 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:52.050654 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:52.050689 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:52.050777 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:52.051586 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:52.051590 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:52.051593 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:09:52.051611 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:52.051628 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:09:52.051636 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:52.051734 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:52.051860 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 11:09:52.051910 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:09:52.055648 25116 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59408:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:09:52.059165 25116 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:09:52.072381 25849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:52.068849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:45821 (local address 127.0.0.1:39922)
0812 11:09:52.069434 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:52.069487 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:52.069597 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:52.070739 (+ 1142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:52.070748 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:52.070828 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:52.071472 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:09:52.071476 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:52.071502 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:09:52.071578 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:52.071978 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:52.071991 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:52.072001 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 11:09:52.072113 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 11:09:52.072192 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":1,"thread_start_us":71,"threads_started":1}
I20260812 11:09:52.072384 25854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:52.069302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:40309 (local address 127.0.0.1:55898)
0812 11:09:52.069796 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:52.069808 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:52.069887 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:52.071037 (+ 1150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:52.071042 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:52.071063 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:52.071617 (+ 554us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:09:52.071619 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:52.071631 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:09:52.071660 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:52.072071 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:52.072077 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:52.072081 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 11:09:52.072132 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 11:09:52.072185 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":126,"threads_started":1}
I20260812 11:09:52.073002 25850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:52.069158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39922 (local address 127.17.44.194:45821)
0812 11:09:52.070213 (+ 1055us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:52.070227 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:52.070255 (+ 28us) server_negotiation.cc:407] Connection header received
0812 11:09:52.070321 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:52.070327 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:52.070549 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:52.070701 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:52.071750 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:52.071757 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:52.071769 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:09:52.071782 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:52.071831 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:09:52.071868 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:52.071987 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:52.072751 (+ 764us) server_negotiation.cc:299] Negotiation successful
0812 11:09:52.072836 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":952,"thread_start_us":127,"threads_started":1}
I20260812 11:09:52.073007 25857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:52.069759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55898 (local address 127.17.44.193:40309)
0812 11:09:52.070600 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:52.070614 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:52.070638 (+ 24us) server_negotiation.cc:407] Connection header received
0812 11:09:52.070692 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:52.070697 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:52.070849 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:52.071278 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:52.071866 (+ 588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:52.071871 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:52.071878 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:09:52.071885 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:52.071916 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:09:52.071963 (+ 47us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:52.072754 (+ 791us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:52.072798 (+ 44us) server_negotiation.cc:299] Negotiation successful
0812 11:09:52.072853 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":740,"thread_start_us":125,"threads_started":1}
I20260812 11:09:52.073400 25855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:52.069668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:42397 (local address 127.0.0.1:58560)
0812 11:09:52.070135 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:52.070147 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:52.070215 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:52.071147 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:52.071151 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:52.071168 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:52.072250 (+ 1082us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:09:52.072252 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:52.072263 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:09:52.072289 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:52.072973 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:52.072976 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:52.072978 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 11:09:52.073233 (+ 255us) client_negotiation.cc:240] Negotiation successful
0812 11:09:52.073273 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":99,"threads_started":1}
I20260812 11:09:52.073421 25858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:52.070112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58560 (local address 127.17.44.195:42397)
0812 11:09:52.070479 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:52.070488 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:52.070506 (+ 18us) server_negotiation.cc:407] Connection header received
0812 11:09:52.070547 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:52.070551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:52.070712 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:52.070842 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:52.072425 (+ 1583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:52.072446 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:52.072461 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:09:52.072474 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:52.072514 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:09:52.072544 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:52.072928 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:52.073215 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 11:09:52.073284 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":176,"threads_started":1}
I20260812 11:09:52.076021 25516 tablet_service.cc:1467] Processing CreateTablet for tablet d76abf8995b842a1ac8720a6fc0994af (DEFAULT_TABLE table=test-table [id=50eee5d7fdb647dcabd0109b6dde19d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:52.076020 25308 tablet_service.cc:1467] Processing CreateTablet for tablet d76abf8995b842a1ac8720a6fc0994af (DEFAULT_TABLE table=test-table [id=50eee5d7fdb647dcabd0109b6dde19d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:52.076098 25726 tablet_service.cc:1467] Processing CreateTablet for tablet d76abf8995b842a1ac8720a6fc0994af (DEFAULT_TABLE table=test-table [id=50eee5d7fdb647dcabd0109b6dde19d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:09:52.077071 25516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d76abf8995b842a1ac8720a6fc0994af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:52.077082 25726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d76abf8995b842a1ac8720a6fc0994af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:52.077081 25308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d76abf8995b842a1ac8720a6fc0994af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:52.086133 25860 tablet_bootstrap.cc:492] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03: Bootstrap starting.
I20260812 11:09:52.086534 25863 tablet_bootstrap.cc:492] T d76abf8995b842a1ac8720a6fc0994af P 2b851795a3e940d6b1b2aa90e5036172: Bootstrap starting.
I20260812 11:09:52.086913 25859 tablet_bootstrap.cc:492] T d76abf8995b842a1ac8720a6fc0994af P 69aaab9af579491e832f4822dddf45d9: Bootstrap starting.
I20260812 11:09:52.088698 25863 tablet_bootstrap.cc:654] T d76abf8995b842a1ac8720a6fc0994af P 2b851795a3e940d6b1b2aa90e5036172: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:52.088714 25860 tablet_bootstrap.cc:654] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:52.089020 25859 tablet_bootstrap.cc:654] T d76abf8995b842a1ac8720a6fc0994af P 69aaab9af579491e832f4822dddf45d9: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:52.089655 25860 log.cc:824] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:52.089756 25859 log.cc:824] T d76abf8995b842a1ac8720a6fc0994af P 69aaab9af579491e832f4822dddf45d9: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:52.089926 25863 log.cc:824] T d76abf8995b842a1ac8720a6fc0994af P 2b851795a3e940d6b1b2aa90e5036172: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:52.092263 25859 tablet_bootstrap.cc:492] T d76abf8995b842a1ac8720a6fc0994af P 69aaab9af579491e832f4822dddf45d9: No bootstrap required, opened a new log
I20260812 11:09:52.092262 25860 tablet_bootstrap.cc:492] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03: No bootstrap required, opened a new log
I20260812 11:09:52.092469 25859 ts_tablet_manager.cc:1397] T d76abf8995b842a1ac8720a6fc0994af P 69aaab9af579491e832f4822dddf45d9: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.003s
I20260812 11:09:52.092469 25860 ts_tablet_manager.cc:1397] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 11:09:52.093145 25863 tablet_bootstrap.cc:492] T d76abf8995b842a1ac8720a6fc0994af P 2b851795a3e940d6b1b2aa90e5036172: No bootstrap required, opened a new log
I20260812 11:09:52.093369 25863 ts_tablet_manager.cc:1397] T d76abf8995b842a1ac8720a6fc0994af P 2b851795a3e940d6b1b2aa90e5036172: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 11:09:52.097194 25860 raft_consensus.cc:348] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69aaab9af579491e832f4822dddf45d9" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 45821 } } peers { permanent_uuid: "2b851795a3e940d6b1b2aa90e5036172" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 40309 } } peers { permanent_uuid: "1e0113b252554fad9b90ddfe84ee0c03" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 42397 } }
I20260812 11:09:52.097194 25859 raft_consensus.cc:348] T d76abf8995b842a1ac8720a6fc0994af P 69aaab9af579491e832f4822dddf45d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69aaab9af579491e832f4822dddf45d9" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 45821 } } peers { permanent_uuid: "2b851795a3e940d6b1b2aa90e5036172" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 40309 } } peers { permanent_uuid: "1e0113b252554fad9b90ddfe84ee0c03" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 42397 } }
I20260812 11:09:52.097479 25859 raft_consensus.cc:374] T d76abf8995b842a1ac8720a6fc0994af P 69aaab9af579491e832f4822dddf45d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:52.097479 25860 raft_consensus.cc:374] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:52.097541 25859 raft_consensus.cc:729] T d76abf8995b842a1ac8720a6fc0994af P 69aaab9af579491e832f4822dddf45d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69aaab9af579491e832f4822dddf45d9, State: Initialized, Role: FOLLOWER
I20260812 11:09:52.097535 25860 raft_consensus.cc:729] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e0113b252554fad9b90ddfe84ee0c03, State: Initialized, Role: FOLLOWER
I20260812 11:09:52.098053 25859 consensus_queue.cc:260] T d76abf8995b842a1ac8720a6fc0994af P 69aaab9af579491e832f4822dddf45d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69aaab9af579491e832f4822dddf45d9" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 45821 } } peers { permanent_uuid: "2b851795a3e940d6b1b2aa90e5036172" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 40309 } } peers { permanent_uuid: "1e0113b252554fad9b90ddfe84ee0c03" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 42397 } }
I20260812 11:09:52.098047 25860 consensus_queue.cc:260] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69aaab9af579491e832f4822dddf45d9" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 45821 } } peers { permanent_uuid: "2b851795a3e940d6b1b2aa90e5036172" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 40309 } } peers { permanent_uuid: "1e0113b252554fad9b90ddfe84ee0c03" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 42397 } }
I20260812 11:09:52.098371 25863 raft_consensus.cc:348] T d76abf8995b842a1ac8720a6fc0994af P 2b851795a3e940d6b1b2aa90e5036172 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69aaab9af579491e832f4822dddf45d9" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 45821 } } peers { permanent_uuid: "2b851795a3e940d6b1b2aa90e5036172" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 40309 } } peers { permanent_uuid: "1e0113b252554fad9b90ddfe84ee0c03" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 42397 } }
I20260812 11:09:52.098658 25863 raft_consensus.cc:374] T d76abf8995b842a1ac8720a6fc0994af P 2b851795a3e940d6b1b2aa90e5036172 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:52.098729 25863 raft_consensus.cc:729] T d76abf8995b842a1ac8720a6fc0994af P 2b851795a3e940d6b1b2aa90e5036172 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b851795a3e940d6b1b2aa90e5036172, State: Initialized, Role: FOLLOWER
I20260812 11:09:52.099180 25859 ts_tablet_manager.cc:1428] T d76abf8995b842a1ac8720a6fc0994af P 69aaab9af579491e832f4822dddf45d9: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 11:09:52.099180 25860 ts_tablet_manager.cc:1428] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 11:09:52.099151 25863 consensus_queue.cc:260] T d76abf8995b842a1ac8720a6fc0994af P 2b851795a3e940d6b1b2aa90e5036172 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69aaab9af579491e832f4822dddf45d9" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 45821 } } peers { permanent_uuid: "2b851795a3e940d6b1b2aa90e5036172" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 40309 } } peers { permanent_uuid: "1e0113b252554fad9b90ddfe84ee0c03" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 42397 } }
I20260812 11:09:52.099684 25614 heartbeater.cc:503] Master 127.17.44.254:43371 was elected leader, sending a full tablet report...
I20260812 11:09:52.099916 25407 heartbeater.cc:503] Master 127.17.44.254:43371 was elected leader, sending a full tablet report...
I20260812 11:09:52.100102 25863 ts_tablet_manager.cc:1428] T d76abf8995b842a1ac8720a6fc0994af P 2b851795a3e940d6b1b2aa90e5036172: Time spent starting tablet: real 0.007s user 0.006s sys 0.003s
I20260812 11:09:52.101331 25835 heartbeater.cc:503] Master 127.17.44.254:43371 was elected leader, sending a full tablet report...
W20260812 11:09:52.125532 25410 tablet.cc:2369] T d76abf8995b842a1ac8720a6fc0994af P 2b851795a3e940d6b1b2aa90e5036172: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:09:52.190490 25869 raft_consensus.cc:482] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:09:52.190857 25869 raft_consensus.cc:504] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69aaab9af579491e832f4822dddf45d9" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 45821 } } peers { permanent_uuid: "2b851795a3e940d6b1b2aa90e5036172" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 40309 } } peers { permanent_uuid: "1e0113b252554fad9b90ddfe84ee0c03" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 42397 } }
I20260812 11:09:52.192901 25869 leader_election.cc:288] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69aaab9af579491e832f4822dddf45d9 (127.17.44.194:45821), 2b851795a3e940d6b1b2aa90e5036172 (127.17.44.193:40309)
I20260812 11:09:52.195367 25850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:52.193384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:38087 (local address 127.17.44.194:45821)
0812 11:09:52.193513 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:52.193518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:52.193535 (+ 17us) server_negotiation.cc:407] Connection header received
0812 11:09:52.193715 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:52.193718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:52.193795 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:52.193903 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:52.194627 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:52.194631 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:52.194633 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:09:52.194648 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:52.194842 (+ 194us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:09:52.194867 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:52.195011 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:52.195150 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 11:09:52.195213 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 11:09:52.195356 25838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:52.193143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:45821 (local address 127.17.44.195:38087)
0812 11:09:52.193358 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:52.193374 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:52.193462 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:52.193924 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:52.193927 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:52.193944 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:52.194461 (+ 517us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:09:52.194464 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:52.194477 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:09:52.194509 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:52.195027 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:52.195032 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:52.195036 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 11:09:52.195144 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 11:09:52.195189 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129}
I20260812 11:09:52.195508 25857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:52.193427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:38079 (local address 127.17.44.193:40309)
0812 11:09:52.193667 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:52.193672 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:52.193952 (+ 280us) server_negotiation.cc:407] Connection header received
0812 11:09:52.194089 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:52.194094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:52.194139 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:52.194224 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:52.195058 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:52.195064 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:52.195069 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:09:52.195086 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:52.195112 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:09:52.195125 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:52.195213 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:52.195337 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 11:09:52.195379 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
I20260812 11:09:52.195544 25874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:52.193143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:40309 (local address 127.17.44.195:38079)
0812 11:09:52.193854 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:52.193867 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:52.193983 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:52.194261 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:52.194263 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:52.194274 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:52.194888 (+ 614us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:09:52.194891 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:52.194905 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:09:52.194937 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:52.195228 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:52.195231 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:52.195234 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 11:09:52.195312 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 11:09:52.195356 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"mutex_wait_us":342,"thread_start_us":141,"threads_started":1}
I20260812 11:09:52.196051 25554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d76abf8995b842a1ac8720a6fc0994af" candidate_uuid: "1e0113b252554fad9b90ddfe84ee0c03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69aaab9af579491e832f4822dddf45d9" is_pre_election: true
I20260812 11:09:52.196197 25338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d76abf8995b842a1ac8720a6fc0994af" candidate_uuid: "1e0113b252554fad9b90ddfe84ee0c03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b851795a3e940d6b1b2aa90e5036172" is_pre_election: true
I20260812 11:09:52.196419 25554 raft_consensus.cc:2454] T d76abf8995b842a1ac8720a6fc0994af P 69aaab9af579491e832f4822dddf45d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e0113b252554fad9b90ddfe84ee0c03 in term 0.
I20260812 11:09:52.196501 25338 raft_consensus.cc:2454] T d76abf8995b842a1ac8720a6fc0994af P 2b851795a3e940d6b1b2aa90e5036172 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e0113b252554fad9b90ddfe84ee0c03 in term 0.
I20260812 11:09:52.196920 25644 leader_election.cc:302] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1e0113b252554fad9b90ddfe84ee0c03, 2b851795a3e940d6b1b2aa90e5036172, 69aaab9af579491e832f4822dddf45d9; no voters:
I20260812 11:09:52.197240 25869 raft_consensus.cc:2781] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:09:52.197333 25869 raft_consensus.cc:482] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:09:52.197386 25869 raft_consensus.cc:3037] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:52.198478 25869 raft_consensus.cc:504] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69aaab9af579491e832f4822dddf45d9" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 45821 } } peers { permanent_uuid: "2b851795a3e940d6b1b2aa90e5036172" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 40309 } } peers { permanent_uuid: "1e0113b252554fad9b90ddfe84ee0c03" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 42397 } }
I20260812 11:09:52.199024 25869 leader_election.cc:288] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03 [CANDIDATE]: Term 1 election: Requested vote from peers 69aaab9af579491e832f4822dddf45d9 (127.17.44.194:45821), 2b851795a3e940d6b1b2aa90e5036172 (127.17.44.193:40309)
I20260812 11:09:52.199528 25338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d76abf8995b842a1ac8720a6fc0994af" candidate_uuid: "1e0113b252554fad9b90ddfe84ee0c03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b851795a3e940d6b1b2aa90e5036172"
I20260812 11:09:52.199662 25338 raft_consensus.cc:3037] T d76abf8995b842a1ac8720a6fc0994af P 2b851795a3e940d6b1b2aa90e5036172 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:52.199769 25554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d76abf8995b842a1ac8720a6fc0994af" candidate_uuid: "1e0113b252554fad9b90ddfe84ee0c03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69aaab9af579491e832f4822dddf45d9"
I20260812 11:09:52.199936 25554 raft_consensus.cc:3037] T d76abf8995b842a1ac8720a6fc0994af P 69aaab9af579491e832f4822dddf45d9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:52.200783 25338 raft_consensus.cc:2454] T d76abf8995b842a1ac8720a6fc0994af P 2b851795a3e940d6b1b2aa90e5036172 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e0113b252554fad9b90ddfe84ee0c03 in term 1.
I20260812 11:09:52.201030 25554 raft_consensus.cc:2454] T d76abf8995b842a1ac8720a6fc0994af P 69aaab9af579491e832f4822dddf45d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e0113b252554fad9b90ddfe84ee0c03 in term 1.
I20260812 11:09:52.201114 25650 leader_election.cc:302] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e0113b252554fad9b90ddfe84ee0c03, 2b851795a3e940d6b1b2aa90e5036172; no voters:
I20260812 11:09:52.201290 25869 raft_consensus.cc:2781] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:52.201525 25869 raft_consensus.cc:686] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03 [term 1 LEADER]: Becoming Leader. State: Replica: 1e0113b252554fad9b90ddfe84ee0c03, State: Running, Role: LEADER
I20260812 11:09:52.201871 25869 consensus_queue.cc:237] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69aaab9af579491e832f4822dddf45d9" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 45821 } } peers { permanent_uuid: "2b851795a3e940d6b1b2aa90e5036172" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 40309 } } peers { permanent_uuid: "1e0113b252554fad9b90ddfe84ee0c03" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 42397 } }
I20260812 11:09:52.205350 25114 catalog_manager.cc:5495] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e0113b252554fad9b90ddfe84ee0c03 (127.17.44.195). New cstate: current_term: 1 leader_uuid: "1e0113b252554fad9b90ddfe84ee0c03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69aaab9af579491e832f4822dddf45d9" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 45821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b851795a3e940d6b1b2aa90e5036172" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 40309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e0113b252554fad9b90ddfe84ee0c03" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 42397 } health_report { overall_health: HEALTHY } } }
I20260812 11:09:52.231401 25858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:52.229635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58562 (local address 127.17.44.195:42397)
0812 11:09:52.229852 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:52.229858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:52.229876 (+ 18us) server_negotiation.cc:407] Connection header received
0812 11:09:52.230030 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:52.230034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:52.230096 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:52.230197 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:52.230963 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:52.230973 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:52.230976 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:09:52.230988 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:52.231006 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:09:52.231014 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:52.231124 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:52.231212 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 11:09:52.231250 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260812 11:09:52.238102 25338 raft_consensus.cc:1261] T d76abf8995b842a1ac8720a6fc0994af P 2b851795a3e940d6b1b2aa90e5036172 [term 1 FOLLOWER]: Refusing update from remote peer 1e0113b252554fad9b90ddfe84ee0c03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:09:52.238101 25554 raft_consensus.cc:1261] T d76abf8995b842a1ac8720a6fc0994af P 69aaab9af579491e832f4822dddf45d9 [term 1 FOLLOWER]: Refusing update from remote peer 1e0113b252554fad9b90ddfe84ee0c03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:09:52.238844 25869 consensus_queue.cc:1035] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b851795a3e940d6b1b2aa90e5036172" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 40309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:09:52.239130 25876 consensus_queue.cc:1035] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69aaab9af579491e832f4822dddf45d9" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 45821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:09:52.248382 25884 mvcc.cc:205] Tried to move back new op lower bound from 7317639136199802880 to 7317639136061722624. Current Snapshot: MvccSnapshot[applied={T|T < 7317639136199802880}]
I20260812 11:09:52.259999 25116 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:09:52.260176 25116 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:09:52.262843 25116 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 58caea7c93504c17a713de996f7143c8: Sending DeleteTablet for 3 replicas of tablet d76abf8995b842a1ac8720a6fc0994af
I20260812 11:09:52.264235 25726 tablet_service.cc:1514] Processing DeleteTablet for tablet d76abf8995b842a1ac8720a6fc0994af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:09:52 UTC) from {username='slave'} at 127.0.0.1:58560
I20260812 11:09:52.264230 25308 tablet_service.cc:1514] Processing DeleteTablet for tablet d76abf8995b842a1ac8720a6fc0994af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:09:52 UTC) from {username='slave'} at 127.0.0.1:55898
I20260812 11:09:52.264232 25516 tablet_service.cc:1514] Processing DeleteTablet for tablet d76abf8995b842a1ac8720a6fc0994af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:09:52 UTC) from {username='slave'} at 127.0.0.1:39922
I20260812 11:09:52.264918 25899 tablet_replica.cc:330] T d76abf8995b842a1ac8720a6fc0994af P 2b851795a3e940d6b1b2aa90e5036172: stopping tablet replica
I20260812 11:09:52.265089 25898 tablet_replica.cc:330] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03: stopping tablet replica
I20260812 11:09:52.265372 25899 raft_consensus.cc:2229] T d76abf8995b842a1ac8720a6fc0994af P 2b851795a3e940d6b1b2aa90e5036172 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:09:52.265547 25898 raft_consensus.cc:2229] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:09:52.265715 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 25211
I20260812 11:09:52.265970 25899 raft_consensus.cc:2258] T d76abf8995b842a1ac8720a6fc0994af P 2b851795a3e940d6b1b2aa90e5036172 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:09:52.266251 25898 raft_consensus.cc:2258] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:09:52.266536 25900 tablet_replica.cc:330] T d76abf8995b842a1ac8720a6fc0994af P 69aaab9af579491e832f4822dddf45d9: stopping tablet replica
I20260812 11:09:52.266852 25900 raft_consensus.cc:2229] T d76abf8995b842a1ac8720a6fc0994af P 69aaab9af579491e832f4822dddf45d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:09:52.267172 25900 raft_consensus.cc:2258] T d76abf8995b842a1ac8720a6fc0994af P 69aaab9af579491e832f4822dddf45d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:09:52.271863 25898 ts_tablet_manager.cc:1905] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:09:52.275775 25898 ts_tablet_manager.cc:1918] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:09:52.275892 25898 log.cc:1196] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/d76abf8995b842a1ac8720a6fc0994af
I20260812 11:09:52.276196 25900 ts_tablet_manager.cc:1905] T d76abf8995b842a1ac8720a6fc0994af P 69aaab9af579491e832f4822dddf45d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:09:52.276261 25898 ts_tablet_manager.cc:1939] T d76abf8995b842a1ac8720a6fc0994af P 1e0113b252554fad9b90ddfe84ee0c03: Deleting consensus metadata
I20260812 11:09:52.277597 25092 catalog_manager.cc:4841] TS 1e0113b252554fad9b90ddfe84ee0c03 (127.17.44.195:42397): tablet d76abf8995b842a1ac8720a6fc0994af (table test-table [id=50eee5d7fdb647dcabd0109b6dde19d8]) successfully deleted
I20260812 11:09:52.279286 25900 ts_tablet_manager.cc:1918] T d76abf8995b842a1ac8720a6fc0994af P 69aaab9af579491e832f4822dddf45d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:09:52.279392 25900 log.cc:1196] T d76abf8995b842a1ac8720a6fc0994af P 69aaab9af579491e832f4822dddf45d9: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/d76abf8995b842a1ac8720a6fc0994af
I20260812 11:09:52.279716 25900 ts_tablet_manager.cc:1939] T d76abf8995b842a1ac8720a6fc0994af P 69aaab9af579491e832f4822dddf45d9: Deleting consensus metadata
I20260812 11:09:52.280905 25092 catalog_manager.cc:4841] TS 69aaab9af579491e832f4822dddf45d9 (127.17.44.194:45821): tablet d76abf8995b842a1ac8720a6fc0994af (table test-table [id=50eee5d7fdb647dcabd0109b6dde19d8]) successfully deleted
W20260812 11:09:52.281311 25094 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.44.193:40309 (error 108)
I20260812 11:09:52.281440 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 25412
I20260812 11:09:52.282503 25855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:52.282134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:40309 (local address 127.0.0.1:55910)
0812 11:09:52.282330 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:52.282411 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.193:40309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260812 11:09:52.282694 25094 catalog_manager.cc:4584] TS 2b851795a3e940d6b1b2aa90e5036172 (127.17.44.193:40309): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d76abf8995b842a1ac8720a6fc0994af: Network error: Client connection negotiation failed: client connection to 127.17.44.193:40309: connect: Connection refused (error 111)
I20260812 11:09:52.297776 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 25617
I20260812 11:09:52.313118 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 25065
2026-08-12T11:09:52Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1170 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 11:09:52.352331 17587 mini_oidc.cc:152] Starting JWKS server
I20260812 11:09:52.354442 17587 webserver.cc:460] Webserver started at https://127.0.0.1:46767/ using document root <none> and password file <none>
I20260812 11:09:52.354704 17587 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 11:09:52.355458 17587 webserver.cc:460] Webserver started at http://127.0.0.1:46727/ using document root <none> and password file <none>
2026-08-12T11:09:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:09:52Z Disabled control of system clock
I20260812 11:09:52.366629 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:35079
--webserver_interface=127.17.44.254
--webserver_port=0
--builtin_ntp_servers=127.17.44.212:41539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:35079
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46767/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786532961723938-17587-0/testchainca.cert with env {}
W20260812 11:09:52.487445 25915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:52.487682 25915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:52.487721 25915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:52.491605 25915 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:09:52.491659 25915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:52.491674 25915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:52.491691 25915 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:09:52.491705 25915 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 11:09:52.491720 25915 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 11:09:52.491734 25915 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46767/jwks.json
I20260812 11:09:52.496884 25915 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:41539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:35079
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:35079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46767/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786532961723938-17587-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:52.497815 25915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:52.499042 25915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:52.507550 25927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:52.507508 25923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:52.507668 25915 server_base.cc:1034] running on GCE node
W20260812 11:09:52.507529 25925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:52.508334 25915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:52.509480 25915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:52.510660 25915 hybrid_clock.cc:648] HybridClock initialized: now 1786532992510629 us; error 46 us; skew 500 ppm
I20260812 11:09:52.512665 25915 webserver.cc:460] Webserver started at http://127.17.44.254:41209/ using document root <none> and password file <none>
I20260812 11:09:52.513259 25915 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:52.513322 25915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:52.513767 25915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:52.515867 25915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "f739ac01974545bb90905b2057ef45b2"
format_stamp: "Formatted at 2026-08-12 11:09:52 on dist-test-slave-drl0"
server_key: "f157c8c2481f38026824fa1b2d000029"
server_key_iv: "295950d2c038eee4e8aa45a1183b024c"
server_key_version: "encryptionkey@0"
I20260812 11:09:52.516445 25915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "f739ac01974545bb90905b2057ef45b2"
format_stamp: "Formatted at 2026-08-12 11:09:52 on dist-test-slave-drl0"
server_key: "f157c8c2481f38026824fa1b2d000029"
server_key_iv: "295950d2c038eee4e8aa45a1183b024c"
server_key_version: "encryptionkey@0"
I20260812 11:09:52.520619 25915 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 11:09:52.523206 25939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:52.524534 25915 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:09:52.524683 25915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "f739ac01974545bb90905b2057ef45b2"
format_stamp: "Formatted at 2026-08-12 11:09:52 on dist-test-slave-drl0"
server_key: "f157c8c2481f38026824fa1b2d000029"
server_key_iv: "295950d2c038eee4e8aa45a1183b024c"
server_key_version: "encryptionkey@0"
I20260812 11:09:52.524806 25915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:52.552215 25915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:52.553396 25915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:52.553622 25915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:52.561751 25915 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:35079
I20260812 11:09:52.561769 26017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:35079 every 8 connection(s)
I20260812 11:09:52.562750 25915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:09:52.563280 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 25915
I20260812 11:09:52.563431 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:09:52.563858 17587 external_mini_cluster.cc:1428] Setting key db7de2e862351228420ed031072a2a03
I20260812 11:09:52.566674 26020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:52.573230 26020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f739ac01974545bb90905b2057ef45b2: Bootstrap starting.
I20260812 11:09:52.575359 26026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:52.566011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43906 (local address 127.17.44.254:35079)
0812 11:09:52.566674 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:52.566689 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:52.566756 (+ 67us) server_negotiation.cc:407] Connection header received
0812 11:09:52.567786 (+ 1030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:52.567824 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:52.568492 (+ 668us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:52.568909 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:52.569383 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:52.570477 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:52.571791 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:52.572373 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:52.572516 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:52.573500 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:52.573505 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:52.573518 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:09:52.573898 (+ 380us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:52.573953 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:09:52.573973 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:52.574057 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:52.574284 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 11:09:52.574587 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":146,"threads_started":1}
I20260812 11:09:52.576085 26020 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f739ac01974545bb90905b2057ef45b2: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:52.577015 26020 log.cc:824] T 00000000000000000000000000000000 P f739ac01974545bb90905b2057ef45b2: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:52.578490 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:09:52.579254 26020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f739ac01974545bb90905b2057ef45b2: No bootstrap required, opened a new log
I20260812 11:09:52.584057 26020 raft_consensus.cc:348] T 00000000000000000000000000000000 P f739ac01974545bb90905b2057ef45b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f739ac01974545bb90905b2057ef45b2" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 35079 } }
I20260812 11:09:52.584383 26020 raft_consensus.cc:374] T 00000000000000000000000000000000 P f739ac01974545bb90905b2057ef45b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:52.584424 26020 raft_consensus.cc:729] T 00000000000000000000000000000000 P f739ac01974545bb90905b2057ef45b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f739ac01974545bb90905b2057ef45b2, State: Initialized, Role: FOLLOWER
I20260812 11:09:52.585134 26020 consensus_queue.cc:260] T 00000000000000000000000000000000 P f739ac01974545bb90905b2057ef45b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f739ac01974545bb90905b2057ef45b2" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 35079 } }
I20260812 11:09:52.585285 26020 raft_consensus.cc:388] T 00000000000000000000000000000000 P f739ac01974545bb90905b2057ef45b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:09:52.585366 26020 raft_consensus.cc:482] T 00000000000000000000000000000000 P f739ac01974545bb90905b2057ef45b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:09:52.585477 26020 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f739ac01974545bb90905b2057ef45b2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:52.587077 26020 raft_consensus.cc:504] T 00000000000000000000000000000000 P f739ac01974545bb90905b2057ef45b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f739ac01974545bb90905b2057ef45b2" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 35079 } }
I20260812 11:09:52.587615 26020 leader_election.cc:302] T 00000000000000000000000000000000 P f739ac01974545bb90905b2057ef45b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f739ac01974545bb90905b2057ef45b2; no voters:
I20260812 11:09:52.588049 26020 leader_election.cc:288] T 00000000000000000000000000000000 P f739ac01974545bb90905b2057ef45b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:09:52.588259 26030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f739ac01974545bb90905b2057ef45b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:52.588647 26030 raft_consensus.cc:686] T 00000000000000000000000000000000 P f739ac01974545bb90905b2057ef45b2 [term 1 LEADER]: Becoming Leader. State: Replica: f739ac01974545bb90905b2057ef45b2, State: Running, Role: LEADER
I20260812 11:09:52.589010 26030 consensus_queue.cc:237] T 00000000000000000000000000000000 P f739ac01974545bb90905b2057ef45b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f739ac01974545bb90905b2057ef45b2" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 35079 } }
I20260812 11:09:52.589846 26020 sys_catalog.cc:564] T 00000000000000000000000000000000 P f739ac01974545bb90905b2057ef45b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:09:52.590684 26036 sys_catalog.cc:455] T 00000000000000000000000000000000 P f739ac01974545bb90905b2057ef45b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f739ac01974545bb90905b2057ef45b2. Latest consensus state: current_term: 1 leader_uuid: "f739ac01974545bb90905b2057ef45b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f739ac01974545bb90905b2057ef45b2" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 35079 } } }
I20260812 11:09:52.590821 26036 sys_catalog.cc:458] T 00000000000000000000000000000000 P f739ac01974545bb90905b2057ef45b2 [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:52.591141 26031 sys_catalog.cc:455] T 00000000000000000000000000000000 P f739ac01974545bb90905b2057ef45b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f739ac01974545bb90905b2057ef45b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f739ac01974545bb90905b2057ef45b2" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 35079 } } }
I20260812 11:09:52.591240 26031 sys_catalog.cc:458] T 00000000000000000000000000000000 P f739ac01974545bb90905b2057ef45b2 [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:52.591362 26039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:09:52.594995 26039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:09:52.601691 26039 catalog_manager.cc:1347] Generated new cluster ID: 2fdc5a341b154e7e946d9a887e29855d
I20260812 11:09:52.601799 26039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:09:52.620687 26039 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:09:52.621644 26039 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:09:52.628273 26039 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f739ac01974545bb90905b2057ef45b2: Generated new TSK 0
I20260812 11:09:52.629441 26039 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 11:09:52.665215 26071 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 11:09:52.665444 26071 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 11:09:52.665795 26026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:52.647562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43914 (local address 127.17.44.254:35079)
0812 11:09:52.647712 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:52.647719 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:52.648115 (+ 396us) server_negotiation.cc:407] Connection header received
0812 11:09:52.648323 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:52.648327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:52.648388 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:52.648497 (+ 109us) server_negotiation.cc:226] Negotiated authn=JWT
0812 11:09:52.648773 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:52.649611 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:52.650509 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:52.650879 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:52.651023 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:52.651387 (+ 364us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 11:09:52.665383 (+ 13996us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46767/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 11:09:52.665646 (+ 263us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43914: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46767/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":60}
W20260812 11:09:52.665887 26026 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43914: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46767/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 11:09:52.667788 26070 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:09:52.647469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:35079 (local address 127.0.0.1:43914)
0812 11:09:52.647906 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:52.648031 (+ 125us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:52.648157 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:52.648528 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:52.648532 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:52.648541 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0812 11:09:52.648664 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:52.648670 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:52.649766 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:52.649769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:52.650392 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:52.650399 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:52.651066 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:52.651070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:52.651143 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:52.651188 (+ 45us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 11:09:52.665819 (+ 14631us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46767/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 11:09:52.666223 (+ 404us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.44.254:35079: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46767/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":122,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.17.44.254:35079: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46767/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 11:09:52.668923 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 25915
2026-08-12T11:09:52Z chronyd exiting
I20260812 11:09:52.714872 17587 test_util.cc:175] -----------------------------------------------
I20260812 11:09:52.715109 17587 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786532961723938-17587-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (365 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 11:09:52.717053 17587 mini_oidc.cc:152] Starting JWKS server
I20260812 11:09:52.718036 17587 webserver.cc:460] Webserver started at https://127.0.0.1:42867/ using document root <none> and password file <none>
I20260812 11:09:52.718183 17587 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 11:09:52.718463 17587 webserver.cc:460] Webserver started at http://127.0.0.1:45255/ using document root <none> and password file <none>
2026-08-12T11:09:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:09:52Z Disabled control of system clock
I20260812 11:09:52.730018 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:35997
--webserver_interface=127.17.44.254
--webserver_port=0
--builtin_ntp_servers=127.17.44.212:44201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:35997
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42867/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786532961723938-17587-0/testchainca.cert with env {}
W20260812 11:09:52.851066 26081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:52.851325 26081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:52.851364 26081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:52.855405 26081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:09:52.855469 26081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:52.855485 26081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:52.855502 26081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:09:52.855518 26081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 11:09:52.855535 26081 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 11:09:52.855548 26081 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42867/jwks.json
I20260812 11:09:52.861425 26081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:44201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:35997
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:35997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42867/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786532961723938-17587-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:52.862543 26081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:52.863643 26081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:52.874884 26088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:52.874918 26090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:52.874884 26087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:52.874951 26081 server_base.cc:1034] running on GCE node
I20260812 11:09:52.875834 26081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:52.876935 26081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:52.878298 26081 hybrid_clock.cc:648] HybridClock initialized: now 1786532992878259 us; error 69 us; skew 500 ppm
I20260812 11:09:52.880838 26081 webserver.cc:460] Webserver started at http://127.17.44.254:39211/ using document root <none> and password file <none>
I20260812 11:09:52.881700 26081 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:52.881786 26081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:52.882041 26081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:52.884439 26081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "f3ceb6371b9f4fc58941bad2e637d229"
format_stamp: "Formatted at 2026-08-12 11:09:52 on dist-test-slave-drl0"
server_key: "e8a62420a5ca24ebd441343cd7b32b4a"
server_key_iv: "52ea603e0a28554dbc2ce5cec409dc4f"
server_key_version: "encryptionkey@0"
I20260812 11:09:52.885051 26081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "f3ceb6371b9f4fc58941bad2e637d229"
format_stamp: "Formatted at 2026-08-12 11:09:52 on dist-test-slave-drl0"
server_key: "e8a62420a5ca24ebd441343cd7b32b4a"
server_key_iv: "52ea603e0a28554dbc2ce5cec409dc4f"
server_key_version: "encryptionkey@0"
I20260812 11:09:52.889417 26081 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 11:09:52.892433 26098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:52.893858 26081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 11:09:52.893975 26081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "f3ceb6371b9f4fc58941bad2e637d229"
format_stamp: "Formatted at 2026-08-12 11:09:52 on dist-test-slave-drl0"
server_key: "e8a62420a5ca24ebd441343cd7b32b4a"
server_key_iv: "52ea603e0a28554dbc2ce5cec409dc4f"
server_key_version: "encryptionkey@0"
I20260812 11:09:52.894124 26081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:52.914412 26081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:52.915215 26081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:52.915418 26081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:52.923177 26081 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:35997
I20260812 11:09:52.923192 26179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:35997 every 8 connection(s)
I20260812 11:09:52.924126 26081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:09:52.927261 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 26081
I20260812 11:09:52.927389 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:09:52.927547 26181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:52.927765 17587 external_mini_cluster.cc:1428] Setting key c28c0e0a8fe00ec1fe6b1e16fd990160
I20260812 11:09:52.937875 26181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3ceb6371b9f4fc58941bad2e637d229: Bootstrap starting.
I20260812 11:09:52.941875 26181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3ceb6371b9f4fc58941bad2e637d229: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:52.942703 26181 log.cc:824] T 00000000000000000000000000000000 P f3ceb6371b9f4fc58941bad2e637d229: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:52.944621 26186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:52.929370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33270 (local address 127.17.44.254:35997)
0812 11:09:52.931503 (+ 2133us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:52.931526 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:52.931574 (+ 48us) server_negotiation.cc:407] Connection header received
0812 11:09:52.932743 (+ 1169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:52.932772 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:52.933296 (+ 524us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:52.933688 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:52.934159 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:52.935818 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:52.937181 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:52.937932 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:52.938218 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:52.939827 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:52.939835 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:52.939854 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:09:52.940439 (+ 585us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:52.940513 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:09:52.940542 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:52.940650 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:52.941112 (+ 462us) server_negotiation.cc:299] Negotiation successful
0812 11:09:52.941445 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1658,"thread_start_us":139,"threads_started":1}
I20260812 11:09:52.946030 26181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3ceb6371b9f4fc58941bad2e637d229: No bootstrap required, opened a new log
I20260812 11:09:52.948050 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:09:52.951305 26181 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3ceb6371b9f4fc58941bad2e637d229 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ceb6371b9f4fc58941bad2e637d229" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 35997 } }
I20260812 11:09:52.951833 26181 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3ceb6371b9f4fc58941bad2e637d229 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:52.951929 26181 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3ceb6371b9f4fc58941bad2e637d229 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3ceb6371b9f4fc58941bad2e637d229, State: Initialized, Role: FOLLOWER
I20260812 11:09:52.952637 26181 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3ceb6371b9f4fc58941bad2e637d229 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ceb6371b9f4fc58941bad2e637d229" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 35997 } }
I20260812 11:09:52.952898 26181 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3ceb6371b9f4fc58941bad2e637d229 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:09:52.952983 26181 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3ceb6371b9f4fc58941bad2e637d229 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:09:52.953203 26181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3ceb6371b9f4fc58941bad2e637d229 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:52.954751 26181 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3ceb6371b9f4fc58941bad2e637d229 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ceb6371b9f4fc58941bad2e637d229" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 35997 } }
I20260812 11:09:52.955221 26181 leader_election.cc:302] T 00000000000000000000000000000000 P f3ceb6371b9f4fc58941bad2e637d229 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3ceb6371b9f4fc58941bad2e637d229; no voters:
I20260812 11:09:52.955644 26181 leader_election.cc:288] T 00000000000000000000000000000000 P f3ceb6371b9f4fc58941bad2e637d229 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:09:52.955775 26190 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3ceb6371b9f4fc58941bad2e637d229 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:52.957851 26190 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3ceb6371b9f4fc58941bad2e637d229 [term 1 LEADER]: Becoming Leader. State: Replica: f3ceb6371b9f4fc58941bad2e637d229, State: Running, Role: LEADER
I20260812 11:09:52.958240 26190 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3ceb6371b9f4fc58941bad2e637d229 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ceb6371b9f4fc58941bad2e637d229" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 35997 } }
I20260812 11:09:52.958815 26181 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3ceb6371b9f4fc58941bad2e637d229 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:09:52.960246 26191 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3ceb6371b9f4fc58941bad2e637d229 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3ceb6371b9f4fc58941bad2e637d229" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ceb6371b9f4fc58941bad2e637d229" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 35997 } } }
I20260812 11:09:52.960395 26191 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3ceb6371b9f4fc58941bad2e637d229 [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:52.960736 26193 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3ceb6371b9f4fc58941bad2e637d229 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3ceb6371b9f4fc58941bad2e637d229. Latest consensus state: current_term: 1 leader_uuid: "f3ceb6371b9f4fc58941bad2e637d229" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ceb6371b9f4fc58941bad2e637d229" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 35997 } } }
I20260812 11:09:52.960866 26193 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3ceb6371b9f4fc58941bad2e637d229 [sys.catalog]: This master's current role is: LEADER
W20260812 11:09:52.964699 26214 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f3ceb6371b9f4fc58941bad2e637d229: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:09:52.964845 26214 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:09:52.964924 26206 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:09:52.965960 26206 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:09:52.974588 26206 catalog_manager.cc:1347] Generated new cluster ID: 96042fd6ca0c44cab0421e70b45fea4b
I20260812 11:09:52.974711 26206 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:09:52.984792 26206 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:09:52.986047 26206 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:09:53.000681 26206 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3ceb6371b9f4fc58941bad2e637d229: Generated new TSK 0
I20260812 11:09:53.001515 26206 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 11:09:53.029971 26227 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 11:09:53.030093 26227 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 11:09:53.030786 26186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:53.013111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33274 (local address 127.17.44.254:35997)
0812 11:09:53.013259 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:53.013264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:53.013430 (+ 166us) server_negotiation.cc:407] Connection header received
0812 11:09:53.013567 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:53.013570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:53.013630 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:53.013754 (+ 124us) server_negotiation.cc:226] Negotiated authn=JWT
0812 11:09:53.014037 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:53.014917 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:53.015953 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:53.016305 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:53.016424 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:53.016753 (+ 329us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 11:09:53.030339 (+ 13586us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42867/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 11:09:53.030626 (+ 287us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33274: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42867/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":58}
W20260812 11:09:53.030865 26186 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33274: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42867/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 11:09:53.030890 26226 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:09:53.012974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:35997 (local address 127.0.0.1:33274)
0812 11:09:53.013340 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:53.013353 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:53.013452 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:53.013801 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:53.013804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:53.013811 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0812 11:09:53.013921 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:53.013926 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:53.015076 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:53.015079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:53.015815 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:53.015825 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:53.016463 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:53.016466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:53.016534 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:53.016558 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 11:09:53.030561 (+ 14003us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42867/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 11:09:53.030717 (+ 156us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.44.254:35997: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42867/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":129,"threads_started":1}
I20260812 11:09:53.031903 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 26081
2026-08-12T11:09:53Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (369 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 11:09:53.085364 17587 mini_oidc.cc:152] Starting JWKS server
I20260812 11:09:53.086416 17587 webserver.cc:460] Webserver started at https://127.0.0.1:36875/ using document root <none> and password file <none>
I20260812 11:09:53.086503 17587 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 11:09:53.086838 17587 webserver.cc:460] Webserver started at http://127.0.0.1:39471/ using document root <none> and password file <none>
2026-08-12T11:09:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:09:53Z Disabled control of system clock
I20260812 11:09:53.097922 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:42523
--webserver_interface=127.17.44.254
--webserver_port=0
--builtin_ntp_servers=127.17.44.212:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:42523
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36875/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 11:09:53.233095 26236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:53.233397 26236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:53.233474 26236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:53.238138 26236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:09:53.238241 26236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:53.238274 26236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:53.238313 26236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:09:53.238350 26236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 11:09:53.238456 26236 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 11:09:53.238549 26236 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36875/jwks.json
I20260812 11:09:53.243760 26236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:42523
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:42523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36875/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:53.244894 26236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:53.246363 26236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:53.256191 26244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:53.256393 26236 server_base.cc:1034] running on GCE node
W20260812 11:09:53.256451 26245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:53.256301 26248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:53.257234 26236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:53.258483 26236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:53.259677 26236 hybrid_clock.cc:648] HybridClock initialized: now 1786532993259599 us; error 93 us; skew 500 ppm
I20260812 11:09:53.261698 26236 webserver.cc:460] Webserver started at http://127.17.44.254:33701/ using document root <none> and password file <none>
I20260812 11:09:53.262413 26236 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:53.262526 26236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:53.262791 26236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:53.264909 26236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "03d6cc1860274386890b6454be486b4b"
format_stamp: "Formatted at 2026-08-12 11:09:53 on dist-test-slave-drl0"
server_key: "d75970ee94a488c9e34b4a3c34ccc586"
server_key_iv: "ab527a478d7079070246af547ec28a9a"
server_key_version: "encryptionkey@0"
I20260812 11:09:53.265507 26236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "03d6cc1860274386890b6454be486b4b"
format_stamp: "Formatted at 2026-08-12 11:09:53 on dist-test-slave-drl0"
server_key: "d75970ee94a488c9e34b4a3c34ccc586"
server_key_iv: "ab527a478d7079070246af547ec28a9a"
server_key_version: "encryptionkey@0"
I20260812 11:09:53.270339 26236 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 11:09:53.273617 26261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:53.275177 26236 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:09:53.275327 26236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "03d6cc1860274386890b6454be486b4b"
format_stamp: "Formatted at 2026-08-12 11:09:53 on dist-test-slave-drl0"
server_key: "d75970ee94a488c9e34b4a3c34ccc586"
server_key_iv: "ab527a478d7079070246af547ec28a9a"
server_key_version: "encryptionkey@0"
I20260812 11:09:53.275461 26236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:53.293753 26236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:53.294487 26236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:53.294711 26236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:53.304764 26236 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:42523
I20260812 11:09:53.304829 26326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:42523 every 8 connection(s)
I20260812 11:09:53.306039 26236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:09:53.309782 26327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:53.315407 26327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03d6cc1860274386890b6454be486b4b: Bootstrap starting.
I20260812 11:09:53.315416 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 26236
I20260812 11:09:53.315586 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:09:53.315881 17587 external_mini_cluster.cc:1428] Setting key fd735ac4be8ea2e3c96160161ee6efac
I20260812 11:09:53.318089 26327 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03d6cc1860274386890b6454be486b4b: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:53.318825 26327 log.cc:824] T 00000000000000000000000000000000 P 03d6cc1860274386890b6454be486b4b: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:53.320605 26327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03d6cc1860274386890b6454be486b4b: No bootstrap required, opened a new log
I20260812 11:09:53.324298 26327 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03d6cc1860274386890b6454be486b4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d6cc1860274386890b6454be486b4b" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42523 } }
I20260812 11:09:53.324543 26327 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03d6cc1860274386890b6454be486b4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:53.324625 26327 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03d6cc1860274386890b6454be486b4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03d6cc1860274386890b6454be486b4b, State: Initialized, Role: FOLLOWER
I20260812 11:09:53.325093 26327 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03d6cc1860274386890b6454be486b4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d6cc1860274386890b6454be486b4b" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42523 } }
I20260812 11:09:53.325253 26327 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03d6cc1860274386890b6454be486b4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:09:53.325322 26327 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03d6cc1860274386890b6454be486b4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:09:53.325444 26327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03d6cc1860274386890b6454be486b4b [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:53.326324 26327 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03d6cc1860274386890b6454be486b4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d6cc1860274386890b6454be486b4b" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42523 } }
I20260812 11:09:53.326644 26332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:53.317242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34016 (local address 127.17.44.254:42523)
0812 11:09:53.317914 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:53.317929 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:53.317965 (+ 36us) server_negotiation.cc:407] Connection header received
0812 11:09:53.318679 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:53.318698 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:53.319159 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:53.319507 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:53.319929 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:53.320959 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:53.321932 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:53.322425 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:53.322596 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:53.323917 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:53.323925 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:53.323948 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:09:53.324577 (+ 629us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:53.324693 (+ 116us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:09:53.324745 (+ 52us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:53.324887 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:53.325069 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 11:09:53.325469 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":145,"threads_started":1}
I20260812 11:09:53.326654 26327 leader_election.cc:302] T 00000000000000000000000000000000 P 03d6cc1860274386890b6454be486b4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03d6cc1860274386890b6454be486b4b; no voters:
I20260812 11:09:53.327095 26327 leader_election.cc:288] T 00000000000000000000000000000000 P 03d6cc1860274386890b6454be486b4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:09:53.327176 26335 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03d6cc1860274386890b6454be486b4b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:53.327378 26335 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03d6cc1860274386890b6454be486b4b [term 1 LEADER]: Becoming Leader. State: Replica: 03d6cc1860274386890b6454be486b4b, State: Running, Role: LEADER
I20260812 11:09:53.327668 26335 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03d6cc1860274386890b6454be486b4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d6cc1860274386890b6454be486b4b" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42523 } }
I20260812 11:09:53.328403 26327 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03d6cc1860274386890b6454be486b4b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:09:53.329276 26336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03d6cc1860274386890b6454be486b4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03d6cc1860274386890b6454be486b4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d6cc1860274386890b6454be486b4b" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42523 } } }
I20260812 11:09:53.329430 26336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03d6cc1860274386890b6454be486b4b [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:53.330015 26338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03d6cc1860274386890b6454be486b4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03d6cc1860274386890b6454be486b4b. Latest consensus state: current_term: 1 leader_uuid: "03d6cc1860274386890b6454be486b4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d6cc1860274386890b6454be486b4b" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42523 } } }
I20260812 11:09:53.330117 26338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03d6cc1860274386890b6454be486b4b [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:53.330395 26348 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:09:53.330824 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:09:53.333822 26348 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:09:53.339402 26348 catalog_manager.cc:1347] Generated new cluster ID: 1475a3caaf8546ed9e87e3fc3ccff172
I20260812 11:09:53.339473 26348 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:09:53.349057 26348 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:09:53.349969 26348 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:09:53.367728 26348 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03d6cc1860274386890b6454be486b4b: Generated new TSK 0
I20260812 11:09:53.368582 26348 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:09:53.388512 26332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:53.382664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34032 (local address 127.17.44.254:42523)
0812 11:09:53.382811 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:53.382815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:53.383026 (+ 211us) server_negotiation.cc:407] Connection header received
0812 11:09:53.383324 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:53.383327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:53.383380 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:53.383490 (+ 110us) server_negotiation.cc:226] Negotiated authn=JWT
0812 11:09:53.383958 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:53.384516 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:53.385789 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:53.386016 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:53.386106 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:53.386533 (+ 427us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 11:09:53.388129 (+ 1596us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36875/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 11:09:53.388359 (+ 230us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34032: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36875/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":59}
W20260812 11:09:53.388585 26332 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34032: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36875/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 11:09:53.390497 26376 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:09:53.382419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42523 (local address 127.0.0.1:34032)
0812 11:09:53.382871 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:53.382890 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:53.383113 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:53.383525 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:53.383528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:53.383539 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0812 11:09:53.383788 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:53.383800 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:53.384638 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:53.384642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:53.385643 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:53.385653 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:53.386176 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:53.386180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:53.386274 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:53.386304 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 11:09:53.388305 (+ 2001us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36875/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 11:09:53.390312 (+ 2007us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.44.254:42523: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36875/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":1810,"thread_start_us":155,"threads_started":1}
I20260812 11:09:53.391155 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 26236
E20260812 11:09:53.416869 26377 webserver.cc:533] Webserver: sslize failed:
E20260812 11:09:53.416956 26377 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T11:09:53Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (364 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T11:09:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:09:53Z Disabled control of system clock
I20260812 11:09:53.460350 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:44009
--webserver_interface=127.17.44.254
--webserver_port=0
--builtin_ntp_servers=127.17.44.212:46761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:44009
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786532961723938-17587-0/testchainca.cert with env {}
W20260812 11:09:53.584946 26382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:53.585281 26382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:53.585424 26382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:53.591161 26382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:09:53.591320 26382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:53.591353 26382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:53.591385 26382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:09:53.591424 26382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 11:09:53.591459 26382 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 11:09:53.591480 26382 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 11:09:53.600696 26382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:46761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:44009
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:44009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786532961723938-17587-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:53.602165 26382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:53.603263 26382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:53.612097 26394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:53.612090 26400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:53.612090 26393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:53.612320 26382 server_base.cc:1034] running on GCE node
I20260812 11:09:53.613044 26382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:53.614162 26382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:53.615463 26382 hybrid_clock.cc:648] HybridClock initialized: now 1786532993615452 us; error 49 us; skew 500 ppm
I20260812 11:09:53.617307 26382 webserver.cc:460] Webserver started at http://127.17.44.254:45039/ using document root <none> and password file <none>
I20260812 11:09:53.617913 26382 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:53.618005 26382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:53.618234 26382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:53.620500 26382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "653c16c666b5405781aede3ee2032b0f"
format_stamp: "Formatted at 2026-08-12 11:09:53 on dist-test-slave-drl0"
server_key: "1f636b1b6663958ecef9fe973dd614a5"
server_key_iv: "c8ae6b561c32d32695ab2b22f7fff295"
server_key_version: "encryptionkey@0"
I20260812 11:09:53.621133 26382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "653c16c666b5405781aede3ee2032b0f"
format_stamp: "Formatted at 2026-08-12 11:09:53 on dist-test-slave-drl0"
server_key: "1f636b1b6663958ecef9fe973dd614a5"
server_key_iv: "c8ae6b561c32d32695ab2b22f7fff295"
server_key_version: "encryptionkey@0"
I20260812 11:09:53.625348 26382 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 11:09:53.628034 26409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:53.629591 26382 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:09:53.629760 26382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "653c16c666b5405781aede3ee2032b0f"
format_stamp: "Formatted at 2026-08-12 11:09:53 on dist-test-slave-drl0"
server_key: "1f636b1b6663958ecef9fe973dd614a5"
server_key_iv: "c8ae6b561c32d32695ab2b22f7fff295"
server_key_version: "encryptionkey@0"
I20260812 11:09:53.629889 26382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:53.646569 26382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:53.647727 26382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:53.647998 26382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:53.656237 26382 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:44009
I20260812 11:09:53.656306 26491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:44009 every 8 connection(s)
I20260812 11:09:53.657323 26382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:09:53.661484 26492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:53.666925 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 26382
I20260812 11:09:53.667138 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:09:53.667428 17587 external_mini_cluster.cc:1428] Setting key 354941314c49bfa4e4d3d4bd17fc3e8f
I20260812 11:09:53.668344 26492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 653c16c666b5405781aede3ee2032b0f: Bootstrap starting.
I20260812 11:09:53.670670 26492 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 653c16c666b5405781aede3ee2032b0f: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:53.671643 26492 log.cc:824] T 00000000000000000000000000000000 P 653c16c666b5405781aede3ee2032b0f: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:53.673856 26492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 653c16c666b5405781aede3ee2032b0f: No bootstrap required, opened a new log
I20260812 11:09:53.677569 26500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:53.668938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36938 (local address 127.17.44.254:44009)
0812 11:09:53.669574 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:53.669589 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:53.669625 (+ 36us) server_negotiation.cc:407] Connection header received
0812 11:09:53.670253 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:53.670270 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:53.670651 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:53.671041 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:53.671405 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:53.672621 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:53.673819 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:53.674340 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:53.674470 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:53.675649 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:53.675654 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:53.675667 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:09:53.676081 (+ 414us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:53.676228 (+ 147us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:09:53.676251 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:53.676336 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:53.676515 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 11:09:53.676829 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":142,"threads_started":1}
I20260812 11:09:53.678053 26492 raft_consensus.cc:348] T 00000000000000000000000000000000 P 653c16c666b5405781aede3ee2032b0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653c16c666b5405781aede3ee2032b0f" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44009 } }
I20260812 11:09:53.678329 26492 raft_consensus.cc:374] T 00000000000000000000000000000000 P 653c16c666b5405781aede3ee2032b0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:53.678378 26492 raft_consensus.cc:729] T 00000000000000000000000000000000 P 653c16c666b5405781aede3ee2032b0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 653c16c666b5405781aede3ee2032b0f, State: Initialized, Role: FOLLOWER
I20260812 11:09:53.678815 26492 consensus_queue.cc:260] T 00000000000000000000000000000000 P 653c16c666b5405781aede3ee2032b0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653c16c666b5405781aede3ee2032b0f" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44009 } }
I20260812 11:09:53.679009 26492 raft_consensus.cc:388] T 00000000000000000000000000000000 P 653c16c666b5405781aede3ee2032b0f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:09:53.679081 26492 raft_consensus.cc:482] T 00000000000000000000000000000000 P 653c16c666b5405781aede3ee2032b0f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:09:53.679205 26492 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 653c16c666b5405781aede3ee2032b0f [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:53.680277 26492 raft_consensus.cc:504] T 00000000000000000000000000000000 P 653c16c666b5405781aede3ee2032b0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653c16c666b5405781aede3ee2032b0f" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44009 } }
I20260812 11:09:53.680622 26492 leader_election.cc:302] T 00000000000000000000000000000000 P 653c16c666b5405781aede3ee2032b0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 653c16c666b5405781aede3ee2032b0f; no voters:
I20260812 11:09:53.680878 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:09:53.681022 26492 leader_election.cc:288] T 00000000000000000000000000000000 P 653c16c666b5405781aede3ee2032b0f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:09:53.681097 26504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 653c16c666b5405781aede3ee2032b0f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:53.681304 26504 raft_consensus.cc:686] T 00000000000000000000000000000000 P 653c16c666b5405781aede3ee2032b0f [term 1 LEADER]: Becoming Leader. State: Replica: 653c16c666b5405781aede3ee2032b0f, State: Running, Role: LEADER
I20260812 11:09:53.681605 26504 consensus_queue.cc:237] T 00000000000000000000000000000000 P 653c16c666b5405781aede3ee2032b0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653c16c666b5405781aede3ee2032b0f" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44009 } }
I20260812 11:09:53.682322 26492 sys_catalog.cc:564] T 00000000000000000000000000000000 P 653c16c666b5405781aede3ee2032b0f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:09:53.683499 26506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 653c16c666b5405781aede3ee2032b0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 653c16c666b5405781aede3ee2032b0f. Latest consensus state: current_term: 1 leader_uuid: "653c16c666b5405781aede3ee2032b0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653c16c666b5405781aede3ee2032b0f" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44009 } } }
I20260812 11:09:53.683527 26505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 653c16c666b5405781aede3ee2032b0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "653c16c666b5405781aede3ee2032b0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653c16c666b5405781aede3ee2032b0f" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44009 } } }
I20260812 11:09:53.683728 26505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 653c16c666b5405781aede3ee2032b0f [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:53.683763 26506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 653c16c666b5405781aede3ee2032b0f [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:53.684175 26514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:09:53.686805 26514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:09:53.694118 26514 catalog_manager.cc:1347] Generated new cluster ID: a8a6ef11c12b4b1fbd92600d894dd4c0
I20260812 11:09:53.694195 26514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:09:53.717725 26514 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:09:53.718673 26514 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:09:53.724692 26514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 653c16c666b5405781aede3ee2032b0f: Generated new TSK 0
I20260812 11:09:53.725332 26514 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:09:54.184278 26500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:53.734619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36940 (local address 127.17.44.254:44009)
0812 11:09:53.734758 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:53.734763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:53.735085 (+ 322us) server_negotiation.cc:407] Connection header received
0812 11:09:53.735207 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:53.735210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:53.735260 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:53.735355 (+ 95us) server_negotiation.cc:226] Negotiated authn=JWT
0812 11:09:53.735653 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:53.736179 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:53.737409 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:53.737631 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:53.737712 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:53.738052 (+ 340us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 11:09:54.183752 (+445700us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 11:09:54.184137 (+ 385us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36940: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":50}
W20260812 11:09:54.184356 26500 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36940: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 11:09:54.184572 26538 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:09:53.734601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:44009 (local address 127.0.0.1:36940)
0812 11:09:53.734952 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:53.734967 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:53.735096 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:53.735376 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:53.735379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:53.735387 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0812 11:09:53.735515 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:53.735522 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:53.736292 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:53.736296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:53.737277 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:53.737288 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:53.737757 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:53.737761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:53.737851 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:53.737879 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 11:09:54.184121 (+446242us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 11:09:54.184323 (+ 202us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.44.254:44009: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":151,"threads_started":1}
I20260812 11:09:54.185225 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 26382
2026-08-12T11:09:54Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (765 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (134 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (140 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:09:54 dist-test-slave-drl0 krb5kdc[26559](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:09:54 dist-test-slave-drl0 krb5kdc[26559](info): set up 2 sockets
Aug 12 11:09:54 dist-test-slave-drl0 krb5kdc[26559](info): commencing operation
krb5kdc: starting...
W20260812 11:09:55.755987 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:09:55 dist-test-slave-drl0 krb5kdc[26559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:09:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:09:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:09:55.893850 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:35023
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:41813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:35023
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:56.018002 26576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:56.018376 26576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:56.018465 26576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:56.026556 26576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:09:56.026643 26576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:56.026679 26576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:56.026710 26576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:09:56.026738 26576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:09:56.036933 26576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:41813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:35023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:35023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:56.038313 26576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:56.039713 26576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:56.050144 26584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:56.050184 26591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:56.050189 26588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:56.050639 26576 server_base.cc:1034] running on GCE node
I20260812 11:09:56.051707 26576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:56.053133 26576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:56.054342 26576 hybrid_clock.cc:648] HybridClock initialized: now 1786532996054307 us; error 43 us; skew 500 ppm
Aug 12 11:09:56 dist-test-slave-drl0 krb5kdc[26559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532996, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:56.061455 26576 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:09:56.062700 26576 webserver.cc:460] Webserver started at http://127.17.44.254:45957/ using document root <none> and password file <none>
I20260812 11:09:56.063465 26576 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:56.063539 26576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:56.063773 26576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:56.067003 26576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "39d603451fff48ce8d0cebfe738f5db4"
format_stamp: "Formatted at 2026-08-12 11:09:56 on dist-test-slave-drl0"
server_key: "bf6e41e6edbfdf0e2585399afda98e47"
server_key_iv: "c1a0bfe6370d653c6e1d6899d4b8c0b3"
server_key_version: "encryptionkey@0"
I20260812 11:09:56.067730 26576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "39d603451fff48ce8d0cebfe738f5db4"
format_stamp: "Formatted at 2026-08-12 11:09:56 on dist-test-slave-drl0"
server_key: "bf6e41e6edbfdf0e2585399afda98e47"
server_key_iv: "c1a0bfe6370d653c6e1d6899d4b8c0b3"
server_key_version: "encryptionkey@0"
I20260812 11:09:56.072213 26576 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 11:09:56.074800 26598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:56.076450 26576 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 11:09:56.076596 26576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "39d603451fff48ce8d0cebfe738f5db4"
format_stamp: "Formatted at 2026-08-12 11:09:56 on dist-test-slave-drl0"
server_key: "bf6e41e6edbfdf0e2585399afda98e47"
server_key_iv: "c1a0bfe6370d653c6e1d6899d4b8c0b3"
server_key_version: "encryptionkey@0"
I20260812 11:09:56.076721 26576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:56.092604 26576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:56.095975 26576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:56.096171 26576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:56.104159 26576 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:35023
I20260812 11:09:56.104187 26677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:35023 every 8 connection(s)
I20260812 11:09:56.105124 26576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:09:56.108923 26678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:56.112061 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 26576
I20260812 11:09:56.112215 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:09:56.112517 17587 external_mini_cluster.cc:1428] Setting key 95446bccc795f5240faf13b0d783a46d
I20260812 11:09:56.119748 26678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39d603451fff48ce8d0cebfe738f5db4: Bootstrap starting.
Aug 12 11:09:56 dist-test-slave-drl0 krb5kdc[26559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:56.123373 26678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39d603451fff48ce8d0cebfe738f5db4: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:56.124365 26678 log.cc:824] T 00000000000000000000000000000000 P 39d603451fff48ce8d0cebfe738f5db4: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:56.126343 26683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:56.113835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58022 (local address 127.17.44.254:35023)
0812 11:09:56.114242 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:56.114265 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:56.114300 (+ 35us) server_negotiation.cc:407] Connection header received
0812 11:09:56.114971 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:56.114998 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:56.115326 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:56.115655 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:56.116449 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:56.117442 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:56.118287 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:56.118752 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:56.118971 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:56.121557 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:56.121562 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:56.121573 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:56.121599 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:56.123365 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:56.123898 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:56.123906 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:56.123917 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:56.124023 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:56.124316 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:56.124319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:56.124321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:56.124770 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:56.124983 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:56.125330 (+ 347us) server_negotiation.cc:299] Negotiation successful
0812 11:09:56.125642 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":140,"threads_started":1}
I20260812 11:09:56.127375 26678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39d603451fff48ce8d0cebfe738f5db4: No bootstrap required, opened a new log
I20260812 11:09:56.132970 26678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39d603451fff48ce8d0cebfe738f5db4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39d603451fff48ce8d0cebfe738f5db4" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 35023 } }
I20260812 11:09:56.133272 26678 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39d603451fff48ce8d0cebfe738f5db4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:56.133344 26678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39d603451fff48ce8d0cebfe738f5db4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39d603451fff48ce8d0cebfe738f5db4, State: Initialized, Role: FOLLOWER
I20260812 11:09:56.133864 26678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39d603451fff48ce8d0cebfe738f5db4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39d603451fff48ce8d0cebfe738f5db4" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 35023 } }
I20260812 11:09:56.134043 26678 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39d603451fff48ce8d0cebfe738f5db4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:09:56.134107 26678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39d603451fff48ce8d0cebfe738f5db4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:09:56.134217 26678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39d603451fff48ce8d0cebfe738f5db4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:56.136355 26678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39d603451fff48ce8d0cebfe738f5db4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39d603451fff48ce8d0cebfe738f5db4" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 35023 } }
I20260812 11:09:56.137172 26678 leader_election.cc:302] T 00000000000000000000000000000000 P 39d603451fff48ce8d0cebfe738f5db4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39d603451fff48ce8d0cebfe738f5db4; no voters:
I20260812 11:09:56.137606 26678 leader_election.cc:288] T 00000000000000000000000000000000 P 39d603451fff48ce8d0cebfe738f5db4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:09:56.137746 26689 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39d603451fff48ce8d0cebfe738f5db4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:56.138007 26689 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39d603451fff48ce8d0cebfe738f5db4 [term 1 LEADER]: Becoming Leader. State: Replica: 39d603451fff48ce8d0cebfe738f5db4, State: Running, Role: LEADER
I20260812 11:09:56.138324 26689 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39d603451fff48ce8d0cebfe738f5db4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39d603451fff48ce8d0cebfe738f5db4" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 35023 } }
I20260812 11:09:56.139156 26678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39d603451fff48ce8d0cebfe738f5db4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:09:56.140065 26691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39d603451fff48ce8d0cebfe738f5db4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39d603451fff48ce8d0cebfe738f5db4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39d603451fff48ce8d0cebfe738f5db4" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 35023 } } }
I20260812 11:09:56.140192 26691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39d603451fff48ce8d0cebfe738f5db4 [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:56.140537 26692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39d603451fff48ce8d0cebfe738f5db4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39d603451fff48ce8d0cebfe738f5db4. Latest consensus state: current_term: 1 leader_uuid: "39d603451fff48ce8d0cebfe738f5db4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39d603451fff48ce8d0cebfe738f5db4" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 35023 } } }
I20260812 11:09:56.140631 26692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39d603451fff48ce8d0cebfe738f5db4 [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:56.141018 26698 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:09:56.144184 26698 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:09:56.150679 26698 catalog_manager.cc:1347] Generated new cluster ID: 1135322bb56644d99fbcf27345f0bc9d
I20260812 11:09:56.150758 26698 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:09:56.177446 26698 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:09:56.178544 26698 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:09:56.185595 26698 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39d603451fff48ce8d0cebfe738f5db4: Generated new TSK 0
I20260812 11:09:56.186388 26698 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
I20260812 11:09:56.218593 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:35023
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:41813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:56.343854 26717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:56.344151 26717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:56.344197 26717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:56.349921 26717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:56.350136 26717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:56.350221 26717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:09:56.356344 26717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:41813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:35023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:56.358022 26717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:56.359501 26717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:56.372048 26732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:56.372151 26717 server_base.cc:1034] running on GCE node
W20260812 11:09:56.372078 26729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:56.372063 26727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:56.372751 26717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:56.373445 26717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:56.374675 26717 hybrid_clock.cc:648] HybridClock initialized: now 1786532996374637 us; error 72 us; skew 500 ppm
Aug 12 11:09:56 dist-test-slave-drl0 krb5kdc[26559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532996, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:56.377889 26717 init.cc:377] Logged in from keytab as kudu/127.17.44.193@KRBTEST.COM (short username kudu)
I20260812 11:09:56.378769 26717 webserver.cc:460] Webserver started at http://127.17.44.193:40587/ using document root <none> and password file <none>
I20260812 11:09:56.379415 26717 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:56.379510 26717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:56.379745 26717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:56.382190 26717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "0b21d10db5db4560b9312413ac6c0fd5"
format_stamp: "Formatted at 2026-08-12 11:09:56 on dist-test-slave-drl0"
server_key: "c532cc4a1e90d8e568de09c20a402ba4"
server_key_iv: "72afb571f4ba7478927e0e91900e785a"
server_key_version: "encryptionkey@0"
I20260812 11:09:56.382778 26717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "0b21d10db5db4560b9312413ac6c0fd5"
format_stamp: "Formatted at 2026-08-12 11:09:56 on dist-test-slave-drl0"
server_key: "c532cc4a1e90d8e568de09c20a402ba4"
server_key_iv: "72afb571f4ba7478927e0e91900e785a"
server_key_version: "encryptionkey@0"
I20260812 11:09:56.386482 26717 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 11:09:56.389003 26743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:56.390336 26717 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 11:09:56.390497 26717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "0b21d10db5db4560b9312413ac6c0fd5"
format_stamp: "Formatted at 2026-08-12 11:09:56 on dist-test-slave-drl0"
server_key: "c532cc4a1e90d8e568de09c20a402ba4"
server_key_iv: "72afb571f4ba7478927e0e91900e785a"
server_key_version: "encryptionkey@0"
I20260812 11:09:56.390617 26717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:56.426023 26717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:56.430356 26717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:56.430588 26717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:56.431259 26717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:56.432144 26717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:56.432219 26717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:56.432291 26717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:56.432344 26717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:56.441931 26717 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:35141
I20260812 11:09:56.441979 26902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:35141 every 8 connection(s)
I20260812 11:09:56.443147 26717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:09:56.446700 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 26717
I20260812 11:09:56.446843 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:09:56.447235 17587 external_mini_cluster.cc:1428] Setting key ef18e66034baf2cf42f423e8206a018e
Aug 12 11:09:56 dist-test-slave-drl0 krb5kdc[26559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532996, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.44.194@KRBTEST.COM; defaulting to no policy
I20260812 11:09:56.460273 26683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:56.445297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:50181 (local address 127.17.44.254:35023)
0812 11:09:56.445446 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:56.445450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:56.446607 (+ 1157us) server_negotiation.cc:407] Connection header received
0812 11:09:56.448580 (+ 1973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:56.448584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:56.448657 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:56.448752 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:56.450297 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:56.450900 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:56.452232 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:56.452485 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:56.452689 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:56.455742 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:56.455745 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:56.455747 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:56.455776 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:56.457551 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:56.458261 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:56.458270 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:56.458271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:56.458358 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:56.458749 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:56.458754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:56.458756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:56.459071 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:56.459190 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:56.459801 (+ 611us) server_negotiation.cc:299] Negotiation successful
0812 11:09:56.459969 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.194@KRBTEST.COM" created.
I20260812 11:09:56.461489 26906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:56.445595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:35023 (local address 127.17.44.193:50181)
0812 11:09:56.446336 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:56.446410 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:56.448324 (+ 1914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:56.448987 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:56.448995 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:56.449469 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:56.450115 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:56.450131 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:56.451099 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:56.451102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:56.452084 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:56.452094 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:56.452816 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:56.452832 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:56.453088 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:56.453910 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:56.453930 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:56.455524 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:56.457741 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:56.457749 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:56.457773 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:56.458123 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:56.458462 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:56.458466 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:56.458469 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:56.458643 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:56.459188 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:56.459198 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:56.459529 (+ 331us) client_negotiation.cc:769] Sending connection context
0812 11:09:56.459760 (+ 231us) client_negotiation.cc:240] Negotiation successful
0812 11:09:56.460346 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":122,"threads_started":1}
I20260812 11:09:56.463570 26903 heartbeater.cc:347] Connected to a master server at 127.17.44.254:35023
I20260812 11:09:56.464020 26903 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:56.464882 26903 heartbeater.cc:511] Master 127.17.44.254:35023 requested a full tablet report, sending...
I20260812 11:09:56.466712 26626 ts_manager.cc:194] Registered new tserver with Master: 0b21d10db5db4560b9312413ac6c0fd5 (127.17.44.193:35141)
I20260812 11:09:56.468930 26626 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.193@KRBTEST.COM'} at 127.17.44.193:50181
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
I20260812 11:09:56.483481 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:35023
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:41813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:56.607570 26911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:56.607843 26911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:56.607889 26911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:56.612252 26911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:56.612320 26911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:56.612380 26911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:09:56.617588 26911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:41813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:35023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:56.618623 26911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:56.619673 26911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:56.630147 26920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:56.630147 26921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:56.630201 26924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:56.630975 26911 server_base.cc:1034] running on GCE node
I20260812 11:09:56.631385 26911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:56.632022 26911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:56.633208 26911 hybrid_clock.cc:648] HybridClock initialized: now 1786532996633188 us; error 31 us; skew 500 ppm
Aug 12 11:09:56 dist-test-slave-drl0 krb5kdc[26559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532996, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:56.636278 26911 init.cc:377] Logged in from keytab as kudu/127.17.44.194@KRBTEST.COM (short username kudu)
I20260812 11:09:56.637279 26911 webserver.cc:460] Webserver started at http://127.17.44.194:40917/ using document root <none> and password file <none>
I20260812 11:09:56.637853 26911 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:56.637951 26911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:56.638191 26911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:56.640437 26911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "c4305c1445ba48c3a019d78242044a21"
format_stamp: "Formatted at 2026-08-12 11:09:56 on dist-test-slave-drl0"
server_key: "e3603fb1af89107d5e8f702e8c85c2c1"
server_key_iv: "e0cc24f72aaa9a9f4ceb41af1ec1b3bb"
server_key_version: "encryptionkey@0"
I20260812 11:09:56.641101 26911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "c4305c1445ba48c3a019d78242044a21"
format_stamp: "Formatted at 2026-08-12 11:09:56 on dist-test-slave-drl0"
server_key: "e3603fb1af89107d5e8f702e8c85c2c1"
server_key_iv: "e0cc24f72aaa9a9f4ceb41af1ec1b3bb"
server_key_version: "encryptionkey@0"
I20260812 11:09:56.645066 26911 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 11:09:56.647655 26937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:56.649016 26911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:09:56.649120 26911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "c4305c1445ba48c3a019d78242044a21"
format_stamp: "Formatted at 2026-08-12 11:09:56 on dist-test-slave-drl0"
server_key: "e3603fb1af89107d5e8f702e8c85c2c1"
server_key_iv: "e0cc24f72aaa9a9f4ceb41af1ec1b3bb"
server_key_version: "encryptionkey@0"
I20260812 11:09:56.649267 26911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:56.660492 26911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:56.663842 26911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:56.664037 26911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:56.664620 26911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:56.665439 26911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:56.665488 26911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:56.665526 26911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:56.665567 26911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:56.675158 26911 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:37849
I20260812 11:09:56.675446 27099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:37849 every 8 connection(s)
I20260812 11:09:56.676100 26911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:09:56.681515 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 26911
I20260812 11:09:56.681670 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:09:56.682011 17587 external_mini_cluster.cc:1428] Setting key c94a159b85a33a5774a55a04a6afe8eb
Aug 12 11:09:56 dist-test-slave-drl0 krb5kdc[26559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532996, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:56.692057 26683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:56.678028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:39277 (local address 127.17.44.254:35023)
0812 11:09:56.678172 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:56.678187 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:56.678952 (+ 765us) server_negotiation.cc:407] Connection header received
0812 11:09:56.680517 (+ 1565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:56.680521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:56.680586 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:56.680685 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:56.682023 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:56.682648 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:56.684050 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:56.684292 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:56.684401 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:56.687592 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:56.687599 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:56.687604 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:56.687635 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:56.689487 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:56.690136 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:56.690142 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:56.690145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:56.690240 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:56.690671 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:56.690676 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:56.690680 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:56.691009 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:56.691133 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:56.691691 (+ 558us) server_negotiation.cc:299] Negotiation successful
0812 11:09:56.691855 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 11:09:56.693341 27103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:56.678302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:35023 (local address 127.17.44.194:39277)
0812 11:09:56.678770 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:56.678811 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:56.680275 (+ 1464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:56.680856 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:56.680863 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:56.681259 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:56.681875 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:56.681892 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:56.682804 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:56.682810 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:56.683922 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:56.683932 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:56.684447 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:56.684452 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:56.684633 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:56.685403 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:56.685429 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:56.687399 (+ 1970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:56.689642 (+ 2243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:56.689648 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:56.689659 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:56.689996 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:56.690357 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:56.690359 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:56.690361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:56.690525 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:56.691162 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:56.691167 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:56.691413 (+ 246us) client_negotiation.cc:769] Sending connection context
0812 11:09:56.691575 (+ 162us) client_negotiation.cc:240] Negotiation successful
0812 11:09:56.692356 (+ 781us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":124,"threads_started":1}
WARNING: no policy specified for kudu/127.17.44.195@KRBTEST.COM; defaulting to no policy
I20260812 11:09:56.694743 27100 heartbeater.cc:347] Connected to a master server at 127.17.44.254:35023
I20260812 11:09:56.695070 27100 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:56.695596 27100 heartbeater.cc:511] Master 127.17.44.254:35023 requested a full tablet report, sending...
I20260812 11:09:56.696748 26626 ts_manager.cc:194] Registered new tserver with Master: c4305c1445ba48c3a019d78242044a21 (127.17.44.194:37849)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.195@KRBTEST.COM" created.
I20260812 11:09:56.697582 26626 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.194@KRBTEST.COM'} at 127.17.44.194:39277
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
I20260812 11:09:56.714854 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:35023
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:41813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:56.845881 27106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:56.846186 27106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:56.846259 27106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:56.850297 27106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:56.850381 27106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:56.850471 27106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:09:56.860203 27106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:41813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:35023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:56.861433 27106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:56.862473 27106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:56.872941 27114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:56.873159 27116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:56.873232 27106 server_base.cc:1034] running on GCE node
W20260812 11:09:56.873159 27120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:56.873769 27106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:56.874378 27106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:56.875700 27106 hybrid_clock.cc:648] HybridClock initialized: now 1786532996875635 us; error 93 us; skew 500 ppm
Aug 12 11:09:56 dist-test-slave-drl0 krb5kdc[26559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532996, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:56.878698 27106 init.cc:377] Logged in from keytab as kudu/127.17.44.195@KRBTEST.COM (short username kudu)
I20260812 11:09:56.880008 27106 webserver.cc:460] Webserver started at http://127.17.44.195:38687/ using document root <none> and password file <none>
I20260812 11:09:56.880661 27106 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:56.880725 27106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:56.880903 27106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:56.883173 27106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "8a9c420774814e3ea9c3fe94d011ef8b"
format_stamp: "Formatted at 2026-08-12 11:09:56 on dist-test-slave-drl0"
server_key: "8914b6cd3896d98f92e8d5f651b2b3f4"
server_key_iv: "ccc9cd48ad868355b881919dce7e9e9a"
server_key_version: "encryptionkey@0"
I20260812 11:09:56.883785 27106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "8a9c420774814e3ea9c3fe94d011ef8b"
format_stamp: "Formatted at 2026-08-12 11:09:56 on dist-test-slave-drl0"
server_key: "8914b6cd3896d98f92e8d5f651b2b3f4"
server_key_iv: "ccc9cd48ad868355b881919dce7e9e9a"
server_key_version: "encryptionkey@0"
I20260812 11:09:56.887650 27106 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 11:09:56.890295 27131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:56.891686 27106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:09:56.891782 27106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "8a9c420774814e3ea9c3fe94d011ef8b"
format_stamp: "Formatted at 2026-08-12 11:09:56 on dist-test-slave-drl0"
server_key: "8914b6cd3896d98f92e8d5f651b2b3f4"
server_key_iv: "ccc9cd48ad868355b881919dce7e9e9a"
server_key_version: "encryptionkey@0"
I20260812 11:09:56.891862 27106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:56.909278 27106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:56.914101 27106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:56.914278 27106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:56.914842 27106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:56.915769 27106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:56.915829 27106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:56.915869 27106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:56.915885 27106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:56.926753 27106 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:46183
I20260812 11:09:56.926783 27297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:46183 every 8 connection(s)
I20260812 11:09:56.927713 27106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:09:56.933137 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 27106
I20260812 11:09:56.933270 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:09:56.933625 17587 external_mini_cluster.cc:1428] Setting key a33e9ce712bcf3a5b8c2ffdc7b9899de
Aug 12 11:09:56 dist-test-slave-drl0 krb5kdc[26559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532996, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:09:56.947503 26683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:56.931052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:39415 (local address 127.17.44.254:35023)
0812 11:09:56.931227 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:56.931233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:56.932849 (+ 1616us) server_negotiation.cc:407] Connection header received
0812 11:09:56.934694 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:56.934697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:56.934742 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:56.934831 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:56.936345 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:56.937175 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:56.939436 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:56.939849 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:56.940003 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:56.943270 (+ 3267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:56.943274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:56.943277 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:56.943303 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:56.945215 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:56.945872 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:56.945875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:56.945877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:56.945965 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:56.946315 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:56.946318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:56.946319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:56.946513 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:56.946621 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:56.947202 (+ 581us) server_negotiation.cc:299] Negotiation successful
0812 11:09:56.947323 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:09:56.948297 27300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:56.931490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:35023 (local address 127.17.44.195:39415)
0812 11:09:56.932681 (+ 1191us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:56.932740 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:56.934468 (+ 1728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:56.935046 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:56.935058 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:56.935460 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:56.936119 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:56.936137 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:56.937314 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:56.937318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:56.939224 (+ 1906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:56.939256 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:56.940194 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:56.940197 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:56.940381 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:56.941139 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:56.941159 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:56.943049 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:56.945349 (+ 2300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:56.945357 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:56.945374 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:56.945713 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:56.946059 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:56.946063 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:56.946065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:56.946208 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:56.946620 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:56.946627 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:56.946885 (+ 258us) client_negotiation.cc:769] Sending connection context
0812 11:09:56.947145 (+ 260us) client_negotiation.cc:240] Negotiation successful
0812 11:09:56.947468 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":938,"thread_start_us":105,"threads_started":1}
I20260812 11:09:56.949919 27298 heartbeater.cc:347] Connected to a master server at 127.17.44.254:35023
I20260812 11:09:56.950131 27298 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:56.950577 27298 heartbeater.cc:511] Master 127.17.44.254:35023 requested a full tablet report, sending...
I20260812 11:09:56.951640 26626 ts_manager.cc:194] Registered new tserver with Master: 8a9c420774814e3ea9c3fe94d011ef8b (127.17.44.195:46183)
I20260812 11:09:56.952211 26626 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.195@KRBTEST.COM'} at 127.17.44.195:39415
I20260812 11:09:56.957690 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/test-admin.keytab.
Aug 12 11:09:56 dist-test-slave-drl0 krb5kdc[26559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:56.975728 17587 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 11:09:56.979238 27306 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:09:56.977470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48684 (local address 127.17.44.193:35141)
0812 11:09:56.977876 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:56.977883 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:56.977910 (+ 27us) server_negotiation.cc:407] Connection header received
0812 11:09:56.978002 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:56.978007 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:56.978208 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:56.978482 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:56.979068 (+ 586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48684: BlockingRecv error: recv got EOF from 127.0.0.1:48684 (error 108)
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":146,"threads_started":1}
W20260812 11:09:56.979285 17587 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/krb5cc' of type FILE
Aug 12 11:09:56 dist-test-slave-drl0 krb5kdc[26559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:56.980607 17587 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 11:09:56 dist-test-slave-drl0 krb5kdc[26559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:56.981582 17587 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 11:09:56 dist-test-slave-drl0 krb5kdc[26559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.193@KRBTEST.COM
I20260812 11:09:56.993113 27306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:56.982553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48690 (local address 127.17.44.193:35141)
0812 11:09:56.982713 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:56.982791 (+ 78us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:56.983007 (+ 216us) server_negotiation.cc:407] Connection header received
0812 11:09:56.983080 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:56.983082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:56.983122 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:56.983191 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:56.983880 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:56.984630 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:56.985820 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:56.986213 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:56.986322 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:56.988826 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:56.988836 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:56.988855 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:56.988881 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:56.990468 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:56.991018 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:56.991024 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:56.991026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:56.991115 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:56.991455 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:56.991459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:56.991460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:56.991842 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:56.992142 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:56.992702 (+ 560us) server_negotiation.cc:299] Negotiation successful
0812 11:09:56.992987 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260812 11:09:56.999944 27306 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:09:56.998461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48704 (local address 127.17.44.193:35141)
0812 11:09:56.998596 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:56.998600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:56.999163 (+ 563us) server_negotiation.cc:407] Connection header received
0812 11:09:56.999215 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:56.999220 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:56.999268 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:56.999367 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:56.999823 (+ 456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48704: BlockingRecv error: recv got EOF from 127.0.0.1:48704 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260812 11:09:57.000057 17587 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/krb5cc' of type FILE
Aug 12 11:09:56 dist-test-slave-drl0 krb5kdc[26559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:57.001255 17587 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 11:09:56 dist-test-slave-drl0 krb5kdc[26559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:57.002601 17587 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 11:09:57 dist-test-slave-drl0 krb5kdc[26559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.193@KRBTEST.COM
I20260812 11:09:57.012797 27306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:57.003534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48714 (local address 127.17.44.193:35141)
0812 11:09:57.003664 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:57.003668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:57.003793 (+ 125us) server_negotiation.cc:407] Connection header received
0812 11:09:57.003948 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:57.003951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:57.003995 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:57.004084 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:57.004621 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:57.005221 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:57.006037 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:57.006276 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:57.006382 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:57.009184 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:57.009187 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:57.009189 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:57.009212 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:57.010792 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:57.011388 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:57.011393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:57.011394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:57.011610 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:57.011927 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:57.011930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:57.011932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:57.012117 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:57.012244 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:57.012476 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 11:09:57.012623 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260812 11:09:57.015532 27306 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:09:57.014484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48722 (local address 127.17.44.193:35141)
0812 11:09:57.014632 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:57.014635 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:57.014662 (+ 27us) server_negotiation.cc:407] Connection header received
0812 11:09:57.014818 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:57.014820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:57.014857 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:57.014980 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:57.015437 (+ 457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48722: BlockingRecv error: recv got EOF from 127.0.0.1:48722 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260812 11:09:57.015573 17587 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786532961723938-17587-0/krb5kdc/krb5cc' of type FILE
Aug 12 11:09:57 dist-test-slave-drl0 krb5kdc[26559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:57.016602 17587 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 11:09:57 dist-test-slave-drl0 krb5kdc[26559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:57.017468 17587 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 11:09:57 dist-test-slave-drl0 krb5kdc[26559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.193@KRBTEST.COM
I20260812 11:09:57.027416 27306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:57.018222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48728 (local address 127.17.44.193:35141)
0812 11:09:57.018381 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:57.018384 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:57.018471 (+ 87us) server_negotiation.cc:407] Connection header received
0812 11:09:57.018556 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:57.018558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:57.018592 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:57.018654 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:57.019743 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:57.020217 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:57.021088 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:57.021305 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:57.021392 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:57.023970 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:57.023973 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:57.023975 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:57.023996 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:57.025583 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:57.026024 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:57.026026 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:57.026027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:57.026100 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:57.026454 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:57.026457 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:57.026459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:57.026674 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:57.026812 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:57.027111 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 11:09:57.027237 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 11:09:57.028709 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 26717
I20260812 11:09:57.046908 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 26911
I20260812 11:09:57.071046 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 27106
I20260812 11:09:57.080066 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 26576
2026-08-12T11:09:57Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2615 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:09:57 dist-test-slave-drl0 krb5kdc[27322](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:09:57 dist-test-slave-drl0 krb5kdc[27322](info): set up 2 sockets
Aug 12 11:09:57 dist-test-slave-drl0 krb5kdc[27322](info): commencing operation
krb5kdc: starting...
W20260812 11:09:58.363385 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:09:58 dist-test-slave-drl0 krb5kdc[27322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:09:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:09:58Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.254.keytab.
Entry for principal oryx/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.254.keytab.
I20260812 11:09:58.502971 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:33165
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.254.keytab
--principal=oryx/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:33165
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:58.623467 27339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:58.623761 27339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:58.623835 27339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:58.627959 27339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:09:58.628065 27339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:58.628105 27339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:58.628165 27339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:09:58.628201 27339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:09:58.633724 27339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:33165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.254.keytab
--principal=oryx/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:33165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:58.634773 27339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:58.635807 27339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:58.645023 27344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:58.645077 27347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:58.645298 27339 server_base.cc:1034] running on GCE node
W20260812 11:09:58.645100 27349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:58.646023 27339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:58.647222 27339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:58.648449 27339 hybrid_clock.cc:648] HybridClock initialized: now 1786532998648418 us; error 46 us; skew 500 ppm
Aug 12 11:09:58 dist-test-slave-drl0 krb5kdc[27322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532998, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:58.652130 27339 init.cc:377] Logged in from keytab as oryx/127.17.44.254@KRBTEST.COM (short username oryx)
I20260812 11:09:58.653455 27339 webserver.cc:460] Webserver started at http://127.17.44.254:42377/ using document root <none> and password file <none>
I20260812 11:09:58.654187 27339 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:58.654295 27339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:58.654552 27339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:58.657217 27339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "fac979aa6d9a4662a500f789ee828ae3"
format_stamp: "Formatted at 2026-08-12 11:09:58 on dist-test-slave-drl0"
server_key: "415ecccb492ca708bd719d08d625a6f8"
server_key_iv: "cf0149e4ba9a5914dc94bd64a07c19ba"
server_key_version: "encryptionkey@0"
I20260812 11:09:58.657914 27339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "fac979aa6d9a4662a500f789ee828ae3"
format_stamp: "Formatted at 2026-08-12 11:09:58 on dist-test-slave-drl0"
server_key: "415ecccb492ca708bd719d08d625a6f8"
server_key_iv: "cf0149e4ba9a5914dc94bd64a07c19ba"
server_key_version: "encryptionkey@0"
I20260812 11:09:58.662130 27339 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 11:09:58.665097 27359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:58.666697 27339 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 11:09:58.666870 27339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "fac979aa6d9a4662a500f789ee828ae3"
format_stamp: "Formatted at 2026-08-12 11:09:58 on dist-test-slave-drl0"
server_key: "415ecccb492ca708bd719d08d625a6f8"
server_key_iv: "cf0149e4ba9a5914dc94bd64a07c19ba"
server_key_version: "encryptionkey@0"
I20260812 11:09:58.667061 27339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:58.682103 27339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:58.686157 27339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:58.686473 27339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:58.694717 27339 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:33165
I20260812 11:09:58.695297 27437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:33165 every 8 connection(s)
I20260812 11:09:58.696460 27339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:09:58.699285 27439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:58.701663 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 27339
I20260812 11:09:58.701818 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:09:58.702268 17587 external_mini_cluster.cc:1428] Setting key 6b74e6e163068d22975bb722fc0f8cd2
I20260812 11:09:58.705591 27439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3: Bootstrap starting.
I20260812 11:09:58.708271 27439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:58.709295 27439 log.cc:824] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:58.711491 27439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3: No bootstrap required, opened a new log
Aug 12 11:09:58 dist-test-slave-drl0 krb5kdc[27322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM
I20260812 11:09:58.715632 27439 raft_consensus.cc:348] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } }
I20260812 11:09:58.715916 27439 raft_consensus.cc:374] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:58.716008 27439 raft_consensus.cc:729] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fac979aa6d9a4662a500f789ee828ae3, State: Initialized, Role: FOLLOWER
I20260812 11:09:58.716495 27439 consensus_queue.cc:260] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } }
I20260812 11:09:58.716665 27439 raft_consensus.cc:388] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:09:58.716754 27439 raft_consensus.cc:482] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:09:58.716863 27439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:09:58.718073 27439 raft_consensus.cc:504] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } }
I20260812 11:09:58.718446 27439 leader_election.cc:302] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fac979aa6d9a4662a500f789ee828ae3; no voters:
I20260812 11:09:58.718786 27439 leader_election.cc:288] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:09:58.718948 27444 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:09:58.719208 27444 raft_consensus.cc:686] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [term 1 LEADER]: Becoming Leader. State: Replica: fac979aa6d9a4662a500f789ee828ae3, State: Running, Role: LEADER
I20260812 11:09:58.719542 27444 consensus_queue.cc:237] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } }
I20260812 11:09:58.720084 27442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:58.704041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34688 (local address 127.17.44.254:33165)
0812 11:09:58.704444 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:58.704460 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:58.704497 (+ 37us) server_negotiation.cc:407] Connection header received
0812 11:09:58.705142 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:58.705159 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:58.705474 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:58.705826 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:58.706714 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:58.707928 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:58.709264 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:58.710059 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:58.710262 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:58.713302 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:58.713307 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:58.713320 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:58.713355 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:58.715649 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:58.716865 (+ 1216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:58.716874 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:58.716882 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:58.717050 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:58.717395 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:58.717406 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:58.717408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:58.717957 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:58.718176 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:58.718525 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 11:09:58.718881 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":125,"threads_started":1}
I20260812 11:09:58.720273 27439 sys_catalog.cc:564] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:09:58.721653 27447 sys_catalog.cc:455] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fac979aa6d9a4662a500f789ee828ae3. Latest consensus state: current_term: 1 leader_uuid: "fac979aa6d9a4662a500f789ee828ae3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } }
I20260812 11:09:58.721798 27447 sys_catalog.cc:458] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:58.721657 27445 sys_catalog.cc:455] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fac979aa6d9a4662a500f789ee828ae3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } }
I20260812 11:09:58.722150 27445 sys_catalog.cc:458] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [sys.catalog]: This master's current role is: LEADER
I20260812 11:09:58.722553 27458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:09:58.725359 27458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:09:58.732710 27458 catalog_manager.cc:1347] Generated new cluster ID: 7332444ac4b746989b256ca4915388fd
I20260812 11:09:58.732805 27458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:09:58.747915 27458 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:09:58.749099 27458 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:09:58.766412 27458 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3: Generated new TSK 0
I20260812 11:09:58.767647 27458 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.193.keytab.
Entry for principal oryx/127.17.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.193.keytab.
I20260812 11:09:58.814191 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:33165
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.193.keytab
--principal=oryx/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:58.942744 27475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:58.943092 27475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:58.943183 27475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:58.947942 27475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:58.948002 27475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:58.948092 27475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:09:58.954777 27475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.193.keytab
--principal=oryx/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:33165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:58.956152 27475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:58.957334 27475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:58.967782 27483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:58.967965 27475 server_base.cc:1034] running on GCE node
W20260812 11:09:58.967797 27488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:58.967777 27485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:58.968520 27475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:58.969116 27475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:58.970350 27475 hybrid_clock.cc:648] HybridClock initialized: now 1786532998970306 us; error 48 us; skew 500 ppm
Aug 12 11:09:58 dist-test-slave-drl0 krb5kdc[27322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532998, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:58.973775 27475 init.cc:377] Logged in from keytab as oryx/127.17.44.193@KRBTEST.COM (short username oryx)
I20260812 11:09:58.974689 27475 webserver.cc:460] Webserver started at http://127.17.44.193:33513/ using document root <none> and password file <none>
I20260812 11:09:58.975343 27475 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:58.975442 27475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:58.975672 27475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:58.977787 27475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "9d36ca09efc643dd9d37dfedd9cd2403"
format_stamp: "Formatted at 2026-08-12 11:09:58 on dist-test-slave-drl0"
server_key: "eef279e2d24992fe4be72c3f046fb53a"
server_key_iv: "c2c99515999d71d65ecc8ce1f40f422f"
server_key_version: "encryptionkey@0"
I20260812 11:09:58.978353 27475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "9d36ca09efc643dd9d37dfedd9cd2403"
format_stamp: "Formatted at 2026-08-12 11:09:58 on dist-test-slave-drl0"
server_key: "eef279e2d24992fe4be72c3f046fb53a"
server_key_iv: "c2c99515999d71d65ecc8ce1f40f422f"
server_key_version: "encryptionkey@0"
I20260812 11:09:58.982026 27475 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260812 11:09:58.984579 27505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:58.986279 27475 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 11:09:58.986441 27475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "9d36ca09efc643dd9d37dfedd9cd2403"
format_stamp: "Formatted at 2026-08-12 11:09:58 on dist-test-slave-drl0"
server_key: "eef279e2d24992fe4be72c3f046fb53a"
server_key_iv: "c2c99515999d71d65ecc8ce1f40f422f"
server_key_version: "encryptionkey@0"
I20260812 11:09:58.986557 27475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:58.998867 27475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:59.002236 27475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:59.002473 27475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:59.003238 27475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:59.004225 27475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:59.004302 27475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:59.004369 27475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:59.004415 27475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:59.015415 27475 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:36081
I20260812 11:09:59.015435 27670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:36081 every 8 connection(s)
I20260812 11:09:59.016281 27475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:09:59.022078 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 27475
I20260812 11:09:59.022229 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:09:59.022661 17587 external_mini_cluster.cc:1428] Setting key c4d853c8f863b8d461cd06152e459f10
Aug 12 11:09:59 dist-test-slave-drl0 krb5kdc[27322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532998, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM
I20260812 11:09:59.033655 27442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:59.018217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:36695 (local address 127.17.44.254:33165)
0812 11:09:59.018374 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:59.018380 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:59.019533 (+ 1153us) server_negotiation.cc:407] Connection header received
0812 11:09:59.020909 (+ 1376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:59.020912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:59.020993 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:59.021081 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:59.022370 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.023174 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.024675 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.025437 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.025625 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:59.028609 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:59.028617 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:59.028623 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:59.028673 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:59.030360 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:59.031149 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:59.031153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:59.031155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:59.031321 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:59.031897 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:59.031903 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:59.031906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:59.032209 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:59.032323 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:59.033272 (+ 949us) server_negotiation.cc:299] Negotiation successful
0812 11:09:59.033476 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 11:09:59.034595 27677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:59.018611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.193:36695)
0812 11:09:59.019361 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:59.019413 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:59.020695 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:59.021231 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:59.021238 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:59.021703 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:59.022213 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:59.022237 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.023336 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.023341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:59.024503 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:59.024515 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.025700 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.025707 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:59.025967 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:59.026752 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:59.026769 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:59.028305 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:59.030509 (+ 2204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:59.030518 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:59.030536 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:59.031002 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:59.031493 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:59.031500 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:59.031505 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:59.031733 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:59.032413 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:59.032423 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:59.032804 (+ 381us) client_negotiation.cc:769] Sending connection context
0812 11:09:59.033055 (+ 251us) client_negotiation.cc:240] Negotiation successful
0812 11:09:59.033448 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":122,"threads_started":1}
WARNING: no policy specified for oryx/127.17.44.194@KRBTEST.COM; defaulting to no policy
I20260812 11:09:59.036118 27671 heartbeater.cc:347] Connected to a master server at 127.17.44.254:33165
I20260812 11:09:59.036377 27671 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:59.036935 27671 heartbeater.cc:511] Master 127.17.44.254:33165 requested a full tablet report, sending...
I20260812 11:09:59.038650 27384 ts_manager.cc:194] Registered new tserver with Master: 9d36ca09efc643dd9d37dfedd9cd2403 (127.17.44.193:36081)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.44.194@KRBTEST.COM" created.
I20260812 11:09:59.040478 27384 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.44.193@KRBTEST.COM'} at 127.17.44.193:36695
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.194.keytab.
Entry for principal oryx/127.17.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.194.keytab.
I20260812 11:09:59.057873 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:33165
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.194.keytab
--principal=oryx/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:59.182058 27680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:59.182317 27680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:59.182360 27680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:59.186610 27680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:59.186674 27680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:59.186733 27680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:09:59.194464 27680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.194.keytab
--principal=oryx/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:33165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:59.195616 27680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:59.196645 27680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:59.207059 27687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:59.207237 27680 server_base.cc:1034] running on GCE node
W20260812 11:09:59.207130 27690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:59.207059 27688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:59.207857 27680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:59.208556 27680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:59.209791 27680 hybrid_clock.cc:648] HybridClock initialized: now 1786532999209756 us; error 54 us; skew 500 ppm
Aug 12 11:09:59 dist-test-slave-drl0 krb5kdc[27322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532999, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:59.212683 27680 init.cc:377] Logged in from keytab as oryx/127.17.44.194@KRBTEST.COM (short username oryx)
I20260812 11:09:59.213554 27680 webserver.cc:460] Webserver started at http://127.17.44.194:36571/ using document root <none> and password file <none>
I20260812 11:09:59.214263 27680 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:59.214375 27680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:59.214622 27680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:59.216722 27680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "084ef8e47f8e4b6d8eac7275ef273315"
format_stamp: "Formatted at 2026-08-12 11:09:59 on dist-test-slave-drl0"
server_key: "17d2918a53171f4fbb442f39df813428"
server_key_iv: "506bae71ecece50896ef514496f02b8c"
server_key_version: "encryptionkey@0"
I20260812 11:09:59.217293 27680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "084ef8e47f8e4b6d8eac7275ef273315"
format_stamp: "Formatted at 2026-08-12 11:09:59 on dist-test-slave-drl0"
server_key: "17d2918a53171f4fbb442f39df813428"
server_key_iv: "506bae71ecece50896ef514496f02b8c"
server_key_version: "encryptionkey@0"
I20260812 11:09:59.221652 27680 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 11:09:59.224833 27698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:59.226357 27680 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:09:59.226505 27680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "084ef8e47f8e4b6d8eac7275ef273315"
format_stamp: "Formatted at 2026-08-12 11:09:59 on dist-test-slave-drl0"
server_key: "17d2918a53171f4fbb442f39df813428"
server_key_iv: "506bae71ecece50896ef514496f02b8c"
server_key_version: "encryptionkey@0"
I20260812 11:09:59.226629 27680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:59.254738 27680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:59.258098 27680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:59.258301 27680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:59.258906 27680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:59.259811 27680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:59.259884 27680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:59.259951 27680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:59.259997 27680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:59.270192 27680 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:32917
I20260812 11:09:59.270219 27875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:32917 every 8 connection(s)
I20260812 11:09:59.271201 27680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:09:59.276880 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 27680
I20260812 11:09:59.277216 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:09:59.277590 17587 external_mini_cluster.cc:1428] Setting key 3df8bba0793d3565916e0513f5ab1e02
Aug 12 11:09:59 dist-test-slave-drl0 krb5kdc[27322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532999, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM
I20260812 11:09:59.288679 27442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:59.273335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:37701 (local address 127.17.44.254:33165)
0812 11:09:59.273575 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:59.273580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:59.274575 (+ 995us) server_negotiation.cc:407] Connection header received
0812 11:09:59.276015 (+ 1440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:59.276018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:59.276065 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:59.276145 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:59.277939 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.278912 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.280377 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.280842 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.281025 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:59.284094 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:59.284101 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:59.284107 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:59.284139 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:59.286185 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:59.286771 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:59.286778 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:59.286781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:59.286885 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:59.287395 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:59.287402 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:59.287405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:59.287664 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:59.287779 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:59.288333 (+ 554us) server_negotiation.cc:299] Negotiation successful
0812 11:09:59.288475 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260812 11:09:59.289542 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:59.273839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.194:37701)
0812 11:09:59.274348 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:59.274384 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:59.275779 (+ 1395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:59.276334 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:59.276340 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:59.276684 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:59.277530 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:59.277580 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.279159 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.279164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:59.280204 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:59.280216 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.281071 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.281075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:59.281310 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:59.282018 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:59.282036 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:59.283913 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:59.286359 (+ 2446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:59.286365 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:59.286377 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:59.286660 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:59.287148 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:59.287151 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:59.287152 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:59.287291 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:59.287811 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:59.287817 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:59.288067 (+ 250us) client_negotiation.cc:769] Sending connection context
0812 11:09:59.288232 (+ 165us) client_negotiation.cc:240] Negotiation successful
0812 11:09:59.288505 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"spinlock_wait_cycles":7552,"thread_start_us":211,"threads_started":1}
WARNING: no policy specified for oryx/127.17.44.195@KRBTEST.COM; defaulting to no policy
I20260812 11:09:59.291260 27878 heartbeater.cc:347] Connected to a master server at 127.17.44.254:33165
I20260812 11:09:59.291766 27878 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.44.195@KRBTEST.COM" created.
I20260812 11:09:59.292603 27878 heartbeater.cc:511] Master 127.17.44.254:33165 requested a full tablet report, sending...
I20260812 11:09:59.293897 27384 ts_manager.cc:194] Registered new tserver with Master: 084ef8e47f8e4b6d8eac7275ef273315 (127.17.44.194:32917)
I20260812 11:09:59.294494 27384 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.44.194@KRBTEST.COM'} at 127.17.44.194:37701
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.195.keytab.
Entry for principal oryx/127.17.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.195.keytab.
I20260812 11:09:59.312727 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:33165
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.195.keytab
--principal=oryx/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:59.433534 27885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:59.433804 27885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:59.433847 27885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:59.439591 27885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:59.439700 27885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:59.439775 27885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:09:59.446910 27885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.195.keytab
--principal=oryx/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:33165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:09:59.448714 27885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:59.450276 27885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:59.463725 27899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:59.463725 27896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:59.463727 27895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:59.463855 27885 server_base.cc:1034] running on GCE node
I20260812 11:09:59.464462 27885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:59.465299 27885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:59.466537 27885 hybrid_clock.cc:648] HybridClock initialized: now 1786532999466508 us; error 40 us; skew 500 ppm
Aug 12 11:09:59 dist-test-slave-drl0 krb5kdc[27322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532999, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:59.469792 27885 init.cc:377] Logged in from keytab as oryx/127.17.44.195@KRBTEST.COM (short username oryx)
I20260812 11:09:59.470732 27885 webserver.cc:460] Webserver started at http://127.17.44.195:33575/ using document root <none> and password file <none>
I20260812 11:09:59.471422 27885 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:59.471487 27885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:59.471704 27885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:59.473819 27885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "ceb2d2ea9c474b06853d1891161296a8"
format_stamp: "Formatted at 2026-08-12 11:09:59 on dist-test-slave-drl0"
server_key: "f1a6a49c7584b580cbd698511375d5f1"
server_key_iv: "fcd15d65de86494432dec068c5dd658d"
server_key_version: "encryptionkey@0"
I20260812 11:09:59.474385 27885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "ceb2d2ea9c474b06853d1891161296a8"
format_stamp: "Formatted at 2026-08-12 11:09:59 on dist-test-slave-drl0"
server_key: "f1a6a49c7584b580cbd698511375d5f1"
server_key_iv: "fcd15d65de86494432dec068c5dd658d"
server_key_version: "encryptionkey@0"
I20260812 11:09:59.478207 27885 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 11:09:59.480804 27909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:59.482071 27885 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:09:59.482213 27885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "ceb2d2ea9c474b06853d1891161296a8"
format_stamp: "Formatted at 2026-08-12 11:09:59 on dist-test-slave-drl0"
server_key: "f1a6a49c7584b580cbd698511375d5f1"
server_key_iv: "fcd15d65de86494432dec068c5dd658d"
server_key_version: "encryptionkey@0"
I20260812 11:09:59.482319 27885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:59.493465 27885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:59.496337 27885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:59.496551 27885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:59.497131 27885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:09:59.497960 27885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:09:59.498009 27885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:59.498080 27885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:09:59.498121 27885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:59.510448 27885 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:34889
I20260812 11:09:59.510704 28078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:34889 every 8 connection(s)
I20260812 11:09:59.511950 27885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:09:59.520697 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 27885
I20260812 11:09:59.520862 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:09:59.521270 17587 external_mini_cluster.cc:1428] Setting key db8c8eb65fae9faae1fcb27b395fffdb
Aug 12 11:09:59 dist-test-slave-drl0 krb5kdc[27322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532999, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM
I20260812 11:09:59.528791 27442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:59.514174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:33387 (local address 127.17.44.254:33165)
0812 11:09:59.514409 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:59.514414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:59.515446 (+ 1032us) server_negotiation.cc:407] Connection header received
0812 11:09:59.517076 (+ 1630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:59.517079 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:59.517136 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:59.517237 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:59.518656 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.519319 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.520590 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.521014 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.521146 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:59.524800 (+ 3654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:59.524806 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:59.524814 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:59.524850 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:59.526539 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:59.527177 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:59.527182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:59.527184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:59.527281 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:59.527639 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:59.527643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:59.527644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:59.527845 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:59.527954 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:59.528478 (+ 524us) server_negotiation.cc:299] Negotiation successful
0812 11:09:59.528604 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260812 11:09:59.529686 28083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:59.514592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.195:33387)
0812 11:09:59.515231 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:59.515285 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:59.516803 (+ 1518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:59.517549 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:59.517559 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:59.517962 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:59.518486 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:59.518502 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.519515 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.519518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:59.520424 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:59.520436 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.521227 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.521230 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:59.521443 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:59.522335 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:59.522369 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:59.524499 (+ 2130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:59.526693 (+ 2194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:59.526699 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:59.526716 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:59.527063 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:59.527385 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:59.527388 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:59.527390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:59.527530 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:59.527970 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:59.527978 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:59.528218 (+ 240us) client_negotiation.cc:769] Sending connection context
0812 11:09:59.528419 (+ 201us) client_negotiation.cc:240] Negotiation successful
0812 11:09:59.528753 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":126,"threads_started":1}
I20260812 11:09:59.531510 28080 heartbeater.cc:347] Connected to a master server at 127.17.44.254:33165
I20260812 11:09:59.531747 28080 heartbeater.cc:464] Registering TS with master...
I20260812 11:09:59.532209 28080 heartbeater.cc:511] Master 127.17.44.254:33165 requested a full tablet report, sending...
I20260812 11:09:59.533439 27384 ts_manager.cc:194] Registered new tserver with Master: ceb2d2ea9c474b06853d1891161296a8 (127.17.44.195:34889)
I20260812 11:09:59.534181 27384 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.44.195@KRBTEST.COM'} at 127.17.44.195:33387
I20260812 11:09:59.536139 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.44.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.44.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.44.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.253.keytab.
Entry for principal oryx/127.17.44.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.253.keytab.
I20260812 11:09:59.572165 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.253:35581
--webserver_interface=127.17.44.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.253.keytab
--principal=oryx/127.17.44.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:33165,127.17.44.253:35581
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:09:59.705699 28088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:09:59.705989 28088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:09:59.706076 28088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:09:59.710465 28088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:09:59.710868 28088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:09:59.710917 28088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:09:59.711015 28088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:09:59.711112 28088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:09:59.718609 28088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:33165,127.17.44.253:35581
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.253.keytab
--principal=oryx/127.17.44.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.253:35581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.44.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
W20260812 11:09:59.718873 28088 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.17.44.254:33165,127.17.44.253:35581'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 11:09:59.719789 28088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:09:59.720851 28088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:59.730299 28096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:59.730311 28100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:59.730311 28097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:59.730353 28088 server_base.cc:1034] running on GCE node
I20260812 11:09:59.731537 28088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:59.732844 28088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:59.734228 28088 hybrid_clock.cc:648] HybridClock initialized: now 1786532999734201 us; error 60 us; skew 500 ppm
Aug 12 11:09:59 dist-test-slave-drl0 krb5kdc[27322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532999, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:59.738461 28088 init.cc:377] Logged in from keytab as oryx/127.17.44.253@KRBTEST.COM (short username oryx)
I20260812 11:09:59.740026 28088 webserver.cc:460] Webserver started at http://127.17.44.253:37171/ using document root <none> and password file <none>
I20260812 11:09:59.740904 28088 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:59.741029 28088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:59.741320 28088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:09:59.743784 28088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/data/instance:
uuid: "395bb06cf1e84d9ba73408995fc72721"
format_stamp: "Formatted at 2026-08-12 11:09:59 on dist-test-slave-drl0"
server_key: "19731ac7b60ef182fbe1ca97dbed233d"
server_key_iv: "5d21f5b12ed8b41176216fa37a07f5c7"
server_key_version: "encryptionkey@0"
I20260812 11:09:59.744418 28088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/wal/instance:
uuid: "395bb06cf1e84d9ba73408995fc72721"
format_stamp: "Formatted at 2026-08-12 11:09:59 on dist-test-slave-drl0"
server_key: "19731ac7b60ef182fbe1ca97dbed233d"
server_key_iv: "5d21f5b12ed8b41176216fa37a07f5c7"
server_key_version: "encryptionkey@0"
I20260812 11:09:59.748325 28088 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 11:09:59.751629 28110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:59.753207 28088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:09:59.753383 28088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/wal
uuid: "395bb06cf1e84d9ba73408995fc72721"
format_stamp: "Formatted at 2026-08-12 11:09:59 on dist-test-slave-drl0"
server_key: "19731ac7b60ef182fbe1ca97dbed233d"
server_key_iv: "5d21f5b12ed8b41176216fa37a07f5c7"
server_key_version: "encryptionkey@0"
I20260812 11:09:59.753505 28088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:59.778671 28088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:59.782392 28088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:09:59.782616 28088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:59.790323 28088 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.253:35581
I20260812 11:09:59.790699 28183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.253:35581 every 8 connection(s)
I20260812 11:09:59.791576 28088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/data/info.pb
I20260812 11:09:59.794458 28184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:09:59.798372 28184 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:09:59.800763 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 28088
Aug 12 11:09:59 dist-test-slave-drl0 krb5kdc[27322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532999, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.253@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM
Aug 12 11:09:59 dist-test-slave-drl0 krb5kdc[27322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.44.253@KRBTEST.COM
I20260812 11:09:59.814800 27442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:59.800610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34702 (local address 127.17.44.254:33165)
0812 11:09:59.800798 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:59.800803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:59.801619 (+ 816us) server_negotiation.cc:407] Connection header received
0812 11:09:59.802643 (+ 1024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:59.802646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:59.802693 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:59.802794 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:59.804109 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.804935 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.806840 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.807200 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.807323 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:59.809976 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:59.809980 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:59.809982 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:59.810055 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:59.812239 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:59.813104 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:59.813107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:59.813109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:59.813189 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:59.813574 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:59.813579 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:59.813584 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:59.813898 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:59.814002 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:59.814472 (+ 470us) server_negotiation.cc:299] Negotiation successful
0812 11:09:59.814617 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 11:09:59.815698 28187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:59.800911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.0.0.1:34702)
0812 11:09:59.801428 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:59.801492 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:59.802441 (+ 949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:59.802957 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:59.802963 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:59.803582 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:59.803963 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:59.803976 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.805253 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.805259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:59.806705 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:59.806715 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.807383 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.807386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:59.807472 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:59.808064 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:59.808080 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:59.809681 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:59.812396 (+ 2715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:59.812407 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:59.812420 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:59.812990 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:59.813287 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:59.813290 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:59.813293 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:59.813472 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:59.814017 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:59.814022 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:59.814222 (+ 200us) client_negotiation.cc:769] Sending connection context
0812 11:09:59.814410 (+ 188us) client_negotiation.cc:240] Negotiation successful
0812 11:09:59.814731 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"spinlock_wait_cycles":1920,"thread_start_us":146,"threads_started":1}
I20260812 11:09:59.815698 28190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:59.801829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45006 (local address 127.17.44.253:35581)
0812 11:09:59.802178 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:59.802189 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:59.802250 (+ 61us) server_negotiation.cc:407] Connection header received
0812 11:09:59.802444 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:59.802459 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:59.802690 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:59.802870 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:59.803951 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.805265 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.806617 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.807224 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.807375 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:59.810439 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:59.810453 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:59.810477 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:59.810651 (+ 174us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:59.812898 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:59.813399 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:59.813404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:59.813405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:59.813481 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:59.813786 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:59.813789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:59.813791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:59.814223 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:59.814447 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:59.814670 (+ 223us) server_negotiation.cc:299] Negotiation successful
0812 11:09:59.814807 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"spinlock_wait_cycles":4480,"thread_start_us":108,"threads_started":1}
I20260812 11:09:59.817091 28184 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 11:09:59 dist-test-slave-drl0 krb5kdc[27322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532999, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.253@KRBTEST.COM for oryx/127.17.44.253@KRBTEST.COM
I20260812 11:09:59.826360 28187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:59.818000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:35581 (local address 127.0.0.1:45012)
0812 11:09:59.818170 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:59.818187 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:59.818270 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:59.818593 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:59.818595 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:59.819172 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:59.819281 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:59.819293 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.820053 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.820056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:59.820708 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:59.820722 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.821265 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.821268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:59.821351 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:59.821848 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:59.821866 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:59.823136 (+ 1270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:59.824871 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:59.824875 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:59.824876 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:59.825081 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:59.825391 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:59.825394 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:59.825395 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:59.825478 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:59.825931 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:59.825934 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:59.826032 (+ 98us) client_negotiation.cc:769] Sending connection context
0812 11:09:59.826135 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 11:09:59.826243 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260812 11:09:59.826467 28190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:59.818083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45012 (local address 127.17.44.253:35581)
0812 11:09:59.818203 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:59.818207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:59.818227 (+ 20us) server_negotiation.cc:407] Connection header received
0812 11:09:59.818411 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:59.818413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:59.818452 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:59.818524 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:59.819424 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.819904 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.820831 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.821077 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.821164 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:59.823261 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:59.823265 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:59.823266 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:59.823287 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:59.824737 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:59.825189 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:59.825192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:59.825193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:59.825266 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:59.825612 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:59.825615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:59.825616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:59.825788 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:59.825891 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:59.826219 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 11:09:59.826314 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 11:09:59.830005 28184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721: Bootstrap starting.
I20260812 11:09:59.832508 28184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721: Neither blocks nor log segments found. Creating new log.
I20260812 11:09:59.833457 28184 log.cc:824] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721: Log is configured to *not* fsync() on all Append() calls
I20260812 11:09:59.835959 28184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721: No bootstrap required, opened a new log
I20260812 11:09:59.839318 28184 raft_consensus.cc:348] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } }
I20260812 11:09:59.839531 28184 raft_consensus.cc:374] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:09:59.839567 28184 raft_consensus.cc:729] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 395bb06cf1e84d9ba73408995fc72721, State: Initialized, Role: FOLLOWER
I20260812 11:09:59.840111 28184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } }
I20260812 11:09:59.840684 28193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } } }
I20260812 11:09:59.840926 28193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:09:59.841441 28184 sys_catalog.cc:564] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 11:09:59.847190 28210 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:09:59.847306 28210 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:09:59.849537 28088 master_runner.cc:428] Detected that this master 395bb06cf1e84d9ba73408995fc72721 is joining an existing cluster
I20260812 11:09:59.849587 28088 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.44.253:35581
I20260812 11:09:59.860407 28190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:59.852149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45024 (local address 127.17.44.253:35581)
0812 11:09:59.852251 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:59.852255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:59.852268 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:09:59.852376 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:59.852389 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:59.852431 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:59.852522 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:59.853297 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.853769 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.854585 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.854825 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.854969 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:59.856525 (+ 1556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:59.856529 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:59.856532 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:59.856557 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:59.858330 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:59.859073 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:59.859078 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:59.859080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:59.859192 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:59.859544 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:59.859547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:59.859549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:59.859756 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:59.859826 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:59.860134 (+ 308us) server_negotiation.cc:299] Negotiation successful
0812 11:09:59.860247 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":31}
I20260812 11:09:59.860776 28221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:59.851708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.0.0.1:34712)
0812 11:09:59.852068 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:59.852084 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:59.852235 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:59.852535 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:59.852538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:59.852756 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:59.852866 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:59.852872 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.853960 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.853963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:59.855233 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:59.855264 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.855940 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.855945 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:59.856089 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:59.856726 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:59.856739 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:59.857170 (+ 431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:59.859099 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:59.859102 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:59.859104 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:59.859387 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:59.859679 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:59.859682 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:59.859684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:59.859773 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:59.860230 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:59.860234 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:59.860324 (+ 90us) client_negotiation.cc:769] Sending connection context
0812 11:09:59.860451 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 11:09:59.860598 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":136,"thread_start_us":106,"threads_started":1}
I20260812 11:09:59.860780 27442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:59.851718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34712 (local address 127.17.44.254:33165)
0812 11:09:59.851855 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:59.851860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:59.852157 (+ 297us) server_negotiation.cc:407] Connection header received
0812 11:09:59.852352 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:59.852358 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:59.852411 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:59.852502 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:59.852994 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.853805 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.855471 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.855773 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.855861 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:59.857311 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:59.857315 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:59.857318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:59.857349 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:59.858978 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:59.859484 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:59.859488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:59.859490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:59.859576 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:59.859865 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:59.859869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:59.859871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:59.860056 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:59.860177 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:59.860527 (+ 350us) server_negotiation.cc:299] Negotiation successful
0812 11:09:59.860639 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:09:59.861497 28220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:59.851773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:35581 (local address 127.0.0.1:45024)
0812 11:09:59.852186 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:59.852198 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:59.852267 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:59.852978 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:59.852980 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:59.853111 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:59.853191 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:59.853195 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.853876 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.853890 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:59.854466 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:59.854473 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.854975 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.854979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:59.855106 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:59.856005 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:59.856016 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:59.856340 (+ 324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:59.858683 (+ 2343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:59.858687 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:59.858688 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:59.858966 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:59.859322 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:59.859326 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:59.859328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:59.859430 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:59.859884 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:59.859887 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:59.859977 (+ 90us) client_negotiation.cc:769] Sending connection context
0812 11:09:59.861192 (+ 1215us) client_negotiation.cc:240] Negotiation successful
0812 11:09:59.861329 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":4,"thread_start_us":123,"threads_started":1}
Aug 12 11:09:59 dist-test-slave-drl0 krb5kdc[27322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532998, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.254@KRBTEST.COM for oryx/127.17.44.253@KRBTEST.COM
I20260812 11:09:59.874657 28190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:59.865323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45026 (local address 127.17.44.253:35581)
0812 11:09:59.865689 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:59.865693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:59.865706 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:09:59.865869 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:59.865872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:59.865918 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:59.866015 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:59.866634 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.867199 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.868053 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.868283 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.868418 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:59.870986 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:59.870989 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:59.870992 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:59.871022 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:59.872747 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:59.873389 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:59.873394 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:59.873397 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:59.873484 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:59.873805 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:59.873808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:59.873809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:59.873956 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:59.874065 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:59.874403 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 11:09:59.874512 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294}
I20260812 11:09:59.874697 28222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:59.865130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:35581 (local address 127.0.0.1:45026)
0812 11:09:59.865540 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:59.865587 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:59.865717 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:59.866050 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:59.866056 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:59.866400 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:59.866516 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:59.866522 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.867310 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.867314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:59.867922 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:59.867931 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.868615 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.868618 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:59.868743 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:59.869309 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:59.869325 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:59.870789 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:59.872886 (+ 2097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:59.872898 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:59.872906 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:59.873243 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:59.873592 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:59.873595 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:59.873597 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:59.873692 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:59.874068 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:59.874077 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:59.874199 (+ 122us) client_negotiation.cc:769] Sending connection context
0812 11:09:59.874351 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 11:09:59.874542 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":97,"threads_started":1}
I20260812 11:09:59.876365 27384 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.44.253:35581: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "395bb06cf1e84d9ba73408995fc72721"
member_type: NON_VOTER
last_known_addr {
host: "127.17.44.253"
port: 35581
}
attrs {
promote: true
}
}
dest_uuid: "fac979aa6d9a4662a500f789ee828ae3"
cas_config_opid_index: -1
I20260812 11:09:59.878269 27384 consensus_queue.cc:237] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: NON_VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: true } }
I20260812 11:09:59.880511 28225 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 395bb06cf1e84d9ba73408995fc72721 (127.17.44.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: NON_VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: true } } }
I20260812 11:09:59.881242 28150 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 0 FOLLOWER]: Advancing to term 1
WARNING: no policy specified for oryx/127.17.44.252@KRBTEST.COM; defaulting to no policy
I20260812 11:09:59.882390 28225 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.44.253:35581
I20260812 11:09:59.882647 28150 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 1 FOLLOWER]: Refusing update from remote peer fac979aa6d9a4662a500f789ee828ae3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 11:09:59.882887 28088 master.cc:537] Master@127.17.44.253:35581 shutting down...
I20260812 11:09:59.883590 28228 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: NON_VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:09:59.884289 28226 sys_catalog.cc:455] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "fac979aa6d9a4662a500f789ee828ae3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: NON_VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: true } } }
I20260812 11:09:59.884416 28226 sys_catalog.cc:458] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [sys.catalog]: This master's current role is: LEADER
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.44.252@KRBTEST.COM" created.
W20260812 11:09:59.885720 27364 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260812 11:09:59.886627 27364 consensus_peers.cc:487] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 -> Peer 395bb06cf1e84d9ba73408995fc72721 (127.17.44.253:35581): Couldn't send request to peer 395bb06cf1e84d9ba73408995fc72721. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 11:09:59.888638 28088 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:09:59.888927 28088 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:09:59.889040 28088 tablet_replica.cc:330] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721: stopping tablet replica
I20260812 11:09:59.905603 28088 master.cc:552] Master@127.17.44.253:35581 shutdown complete.
I20260812 11:09:59.905668 28088 master_runner.cc:305] Clearing existing system tablet
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.44.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.252.keytab.
Entry for principal oryx/127.17.44.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.252.keytab.
I20260812 11:09:59.905933 28088 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:59.906002 28088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:59.907320 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.252:33565
--webserver_interface=127.17.44.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.252.keytab
--principal=oryx/127.17.44.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:33165,127.17.44.253:35581,127.17.44.252:33565
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:09:59.907949 28088 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 11:09:59.909073 28234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:59.909327 28088 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:59.909473 28088 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 11:09:59.909514 28088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/wal
uuid: "395bb06cf1e84d9ba73408995fc72721"
format_stamp: "Formatted at 2026-08-12 11:09:59 on dist-test-slave-drl0"
server_key: "19731ac7b60ef182fbe1ca97dbed233d"
server_key_iv: "5d21f5b12ed8b41176216fa37a07f5c7"
server_key_version: "encryptionkey@0"
I20260812 11:09:59.912005 28088 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:09:59.914978 28088 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 11:09:59.915153 28088 log.cc:1196] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 11:09:59.915475 28088 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721: Deleting consensus metadata
I20260812 11:09:59.915693 28088 master_runner.cc:315] Copying system tablet from 127.17.44.254:33165
I20260812 11:09:59.917323 28088 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721: tablet copy: Beginning tablet copy session from remote peer at address 127.17.44.254:33165
I20260812 11:09:59.927541 28241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:59.918097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.0.0.1:34722)
0812 11:09:59.918538 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0812 11:09:59.918551 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:09:59.918724 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:09:59.919075 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:09:59.919078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:09:59.919326 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:09:59.919451 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:59.919457 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.920565 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.920570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:59.922149 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:09:59.922162 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.922668 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.922672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:09:59.922769 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:59.923430 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:09:59.923460 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:09:59.923821 (+ 361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:09:59.925786 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:59.925790 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:59.925792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:59.926097 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:59.926423 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:09:59.926425 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:09:59.926427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:09:59.926513 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:09:59.926918 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:09:59.926921 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:09:59.927053 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 11:09:59.927170 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 11:09:59.927332 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":103,"threads_started":1}
I20260812 11:09:59.927532 27442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:09:59.918231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34722 (local address 127.17.44.254:33165)
0812 11:09:59.918412 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 11:09:59.918416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:09:59.918651 (+ 235us) server_negotiation.cc:407] Connection header received
0812 11:09:59.918856 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:09:59.918858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:09:59.918910 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:09:59.919040 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:09:59.919576 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.920371 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.922299 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:09:59.922536 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:09:59.922659 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:09:59.923989 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:09:59.923993 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:09:59.923995 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:09:59.924018 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:09:59.925646 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:59.926223 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:59.926226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:59.926228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:59.926312 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:09:59.926610 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:09:59.926614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:09:59.926615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:09:59.926806 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:09:59.926924 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 11:09:59.927226 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 11:09:59.927344 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 11:09:59.928629 27415 tablet_copy_service.cc:140] P fac979aa6d9a4662a500f789ee828ae3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 395bb06cf1e84d9ba73408995fc72721 ({username='oryx', principal='oryx/127.17.44.253@KRBTEST.COM'} at 127.0.0.1:34722)
I20260812 11:09:59.928829 27415 tablet_copy_service.cc:161] P fac979aa6d9a4662a500f789ee828ae3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 395bb06cf1e84d9ba73408995fc72721 at {username='oryx', principal='oryx/127.17.44.253@KRBTEST.COM'} at 127.0.0.1:34722: session id = 395bb06cf1e84d9ba73408995fc72721-00000000000000000000000000000000
I20260812 11:09:59.930586 27415 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 11:09:59.932507 28088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 11:09:59.934741 28088 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721: tablet copy: Starting download of 0 data blocks...
I20260812 11:09:59.935029 28088 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721: tablet copy: Starting download of 1 WAL segments...
I20260812 11:09:59.937134 28088 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 11:09:59.939073 27415 tablet_copy_service.cc:342] P fac979aa6d9a4662a500f789ee828ae3: Request end of tablet copy session 395bb06cf1e84d9ba73408995fc72721-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.44.253@KRBTEST.COM'} at 127.0.0.1:34722
I20260812 11:09:59.939210 27415 tablet_copy_service.cc:433] P fac979aa6d9a4662a500f789ee828ae3: ending tablet copy session 395bb06cf1e84d9ba73408995fc72721-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 395bb06cf1e84d9ba73408995fc72721
I20260812 11:09:59.947229 28088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:09:59.949088 28242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:59.952570 28244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:09:59.953056 28246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:09:59.953213 28088 server_base.cc:1034] running on GCE node
I20260812 11:09:59.953454 28088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:09:59.953858 28088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:09:59.955003 28088 hybrid_clock.cc:648] HybridClock initialized: now 1786532999954991 us; error 26 us; skew 500 ppm
Aug 12 11:09:59 dist-test-slave-drl0 krb5kdc[27322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532999, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:09:59.957321 28088 init.cc:377] Logged in from keytab as oryx/127.17.44.253@KRBTEST.COM (short username oryx)
I20260812 11:09:59.957823 28088 webserver.cc:460] Webserver started at http://127.17.44.253:33213/ using document root <none> and password file <none>
I20260812 11:09:59.958043 28088 fs_manager.cc:359] Metadata directory not provided
I20260812 11:09:59.958101 28088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:09:59.959282 28088 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 11:09:59.960314 28254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:09:59.960551 28088 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 11:09:59.960647 28088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/wal
uuid: "395bb06cf1e84d9ba73408995fc72721"
format_stamp: "Formatted at 2026-08-12 11:09:59 on dist-test-slave-drl0"
server_key: "19731ac7b60ef182fbe1ca97dbed233d"
server_key_iv: "5d21f5b12ed8b41176216fa37a07f5c7"
server_key_version: "encryptionkey@0"
I20260812 11:09:59.960753 28088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:09:59.976614 28088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:09:59.979763 28088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:09:59.986173 28088 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.253:35581
I20260812 11:09:59.986850 28088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-1/data/info.pb
I20260812 11:09:59.988528 28332 sys_catalog.cc:263] Verifying existing consensus state
I20260812 11:09:59.989385 28332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721: Bootstrap starting.
I20260812 11:09:59.991083 28330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.253:35581 every 8 connection(s)
I20260812 11:10:00.000710 28332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 11:10:00.001359 28332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721: Bootstrap complete.
I20260812 11:10:00.002048 28332 raft_consensus.cc:348] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: NON_VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: true } }
I20260812 11:10:00.002199 28332 raft_consensus.cc:729] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 395bb06cf1e84d9ba73408995fc72721, State: Initialized, Role: LEARNER
I20260812 11:10:00.002323 28332 consensus_queue.cc:260] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: NON_VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: true } }
I20260812 11:10:00.002725 28336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: NON_VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: true } } }
I20260812 11:10:00.002908 28336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [sys.catalog]: This master's current role is: LEARNER
I20260812 11:10:00.002924 28332 sys_catalog.cc:564] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:10:00.006562 28348 catalog_manager.cc:1259] Loaded cluster ID: 7332444ac4b746989b256ca4915388fd
I20260812 11:10:00.006665 28348 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721: loading cluster ID for follower catalog manager: success
I20260812 11:10:00.008499 28348 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721: acquiring CA information for follower catalog manager: success
I20260812 11:10:00.010424 28348 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 11:10:00.043386 27671 heartbeater.cc:503] Master 127.17.44.254:33165 was elected leader, sending a full tablet report...
W20260812 11:10:00.086082 28235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:00.086470 28235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:00.086588 28235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:00.092933 28235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:10:00.093034 28235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:00.093075 28235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:00.093112 28235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:10:00.093148 28235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:10:00.099229 28235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:33165,127.17.44.253:35581,127.17.44.252:33565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.252.keytab
--principal=oryx/127.17.44.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.252:33565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.44.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:00.100481 28235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:00.101583 28235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:00.111474 28359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:00.111553 28358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:00.111737 28235 server_base.cc:1034] running on GCE node
W20260812 11:10:00.111617 28362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:00.112430 28235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:00.113601 28235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:00.114874 28235 hybrid_clock.cc:648] HybridClock initialized: now 1786533000114837 us; error 34 us; skew 500 ppm
Aug 12 11:10:00 dist-test-slave-drl0 krb5kdc[27322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533000, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:10:00.119063 28235 init.cc:377] Logged in from keytab as oryx/127.17.44.252@KRBTEST.COM (short username oryx)
I20260812 11:10:00.120321 28235 webserver.cc:460] Webserver started at http://127.17.44.252:33911/ using document root <none> and password file <none>
I20260812 11:10:00.121199 28235 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:00.121410 28235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:00.121711 28235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:10:00.124022 28235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/data/instance:
uuid: "57d679a7e23e4081b1f929452563ca04"
format_stamp: "Formatted at 2026-08-12 11:10:00 on dist-test-slave-drl0"
server_key: "7cc0a9b16deac62409ceab77c756f283"
server_key_iv: "328b26eb4599613f42b54a63638850c6"
server_key_version: "encryptionkey@0"
I20260812 11:10:00.124615 28235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/wal/instance:
uuid: "57d679a7e23e4081b1f929452563ca04"
format_stamp: "Formatted at 2026-08-12 11:10:00 on dist-test-slave-drl0"
server_key: "7cc0a9b16deac62409ceab77c756f283"
server_key_iv: "328b26eb4599613f42b54a63638850c6"
server_key_version: "encryptionkey@0"
I20260812 11:10:00.128603 28235 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 11:10:00.132331 28376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:00.133939 28235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:10:00.134119 28235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/wal
uuid: "57d679a7e23e4081b1f929452563ca04"
format_stamp: "Formatted at 2026-08-12 11:10:00 on dist-test-slave-drl0"
server_key: "7cc0a9b16deac62409ceab77c756f283"
server_key_iv: "328b26eb4599613f42b54a63638850c6"
server_key_version: "encryptionkey@0"
I20260812 11:10:00.134251 28235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:00.153283 28235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:00.157662 28235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:00.157930 28235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:00.169106 28235 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.252:33565
I20260812 11:10:00.169495 28450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.252:33565 every 8 connection(s)
I20260812 11:10:00.170362 28235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/data/info.pb
I20260812 11:10:00.173946 28451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:00.178473 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 28235
I20260812 11:10:00.180125 28451 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 11:10:00 dist-test-slave-drl0 krb5kdc[27322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533000, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.252@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM
Aug 12 11:10:00 dist-test-slave-drl0 krb5kdc[27322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.44.252@KRBTEST.COM
I20260812 11:10:00.194617 27442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.182319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34724 (local address 127.17.44.254:33165)
0812 11:10:00.182468 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:00.182472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:00.182816 (+ 344us) server_negotiation.cc:407] Connection header received
0812 11:10:00.183227 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:00.183230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:00.183295 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:00.183384 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:00.184201 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.184735 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.185863 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.186136 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.186249 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.189375 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:00.189379 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:00.189382 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:00.189408 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:00.191679 (+ 2271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.192205 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.192211 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.192213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.192308 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.192979 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.192983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.192985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.193247 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:00.193340 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:00.194061 (+ 721us) server_negotiation.cc:299] Negotiation successful
0812 11:10:00.194192 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 11:10:00.195322 28455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.180925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34086 (local address 127.17.44.252:33565)
0812 11:10:00.181625 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:00.181696 (+ 71us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:00.181744 (+ 48us) server_negotiation.cc:407] Connection header received
0812 11:10:00.182719 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:00.182741 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:00.183053 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:00.183290 (+ 237us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:00.184268 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.185264 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.186188 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.186658 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.186782 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.189924 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:00.189934 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:00.189950 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:00.189977 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:00.192428 (+ 2451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.192990 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.192997 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.192999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.193081 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.193504 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.193507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.193508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.193767 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:00.194015 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:00.194224 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 11:10:00.194623 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":417,"thread_start_us":205,"threads_started":1}
I20260812 11:10:00.195314 28456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.182122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.0.0.1:34724)
0812 11:10:00.182677 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:00.182723 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:00.183039 (+ 316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:00.183448 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:00.183454 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:00.183801 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:00.184055 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.184068 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.184867 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.184870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.185696 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.185706 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.186295 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.186299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.186694 (+ 395us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.187426 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:00.187451 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:00.189227 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:00.191807 (+ 2580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.191815 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.191823 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.192092 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.192546 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.192551 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.192555 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.192800 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.193382 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:00.193393 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:00.193757 (+ 364us) client_negotiation.cc:769] Sending connection context
0812 11:10:00.193978 (+ 221us) client_negotiation.cc:240] Negotiation successful
0812 11:10:00.194306 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":57,"thread_start_us":214,"threads_started":1}
I20260812 11:10:00.196592 28451 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 11:10:00 dist-test-slave-drl0 krb5kdc[27322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533000, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.252@KRBTEST.COM for oryx/127.17.44.253@KRBTEST.COM
I20260812 11:10:00.206455 28456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.197114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:35581 (local address 127.0.0.1:45028)
0812 11:10:00.197523 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:00.197536 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:00.197601 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:00.198105 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:00.198108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:00.198257 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:00.198347 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.198353 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.199443 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.199446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.199969 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.199977 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.200485 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.200488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.200576 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.201073 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:00.201085 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:00.202297 (+ 1212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:00.204450 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.204453 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.204454 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.204649 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.205114 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.205119 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.205121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.205292 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.205802 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:00.205805 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:00.206034 (+ 229us) client_negotiation.cc:769] Sending connection context
0812 11:10:00.206134 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 11:10:00.206309 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332}
I20260812 11:10:00.206645 28457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.197246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45028 (local address 127.17.44.253:35581)
0812 11:10:00.197830 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:00.197834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:00.197856 (+ 22us) server_negotiation.cc:407] Connection header received
0812 11:10:00.197902 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:00.197904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:00.197971 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:00.198053 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:00.198514 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.199251 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.200123 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.200347 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.200438 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.202451 (+ 2013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:00.202454 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:00.202456 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:00.202478 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:00.204333 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.204766 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.204769 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.204770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.204850 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.205473 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.205476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.205477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.205659 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:00.205753 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:00.206239 (+ 486us) server_negotiation.cc:299] Negotiation successful
0812 11:10:00.206431 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":133,"threads_started":1}
I20260812 11:10:00.207625 28451 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 11:10:00 dist-test-slave-drl0 krb5kdc[27322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533000, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.252@KRBTEST.COM for oryx/127.17.44.252@KRBTEST.COM
I20260812 11:10:00.218598 28455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.208405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34092 (local address 127.17.44.252:33565)
0812 11:10:00.208551 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:00.208554 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:00.208568 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:10:00.208699 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:00.208701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:00.208747 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:00.208814 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:00.209222 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.209718 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.211402 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.211638 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.212026 (+ 388us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.214636 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:00.214639 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:00.214641 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:00.214664 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:00.216254 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.216668 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.216672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.216674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.216757 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.217152 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.217157 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.217158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.217439 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:00.217582 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:00.217950 (+ 368us) server_negotiation.cc:299] Negotiation successful
0812 11:10:00.218185 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 11:10:00.218580 28456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.208204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:33565 (local address 127.0.0.1:34092)
0812 11:10:00.208521 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:00.208533 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:00.208594 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:00.208861 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:00.208864 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:00.209001 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:00.209072 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.209077 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.209844 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.209847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.211272 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.211283 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.212040 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.212046 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.212148 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.212872 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:00.212886 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:00.214476 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:00.216381 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.216384 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.216385 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.216572 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.216872 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.216875 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.216876 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.216960 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.217595 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:00.217597 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:00.217687 (+ 90us) client_negotiation.cc:769] Sending connection context
0812 11:10:00.217889 (+ 202us) client_negotiation.cc:240] Negotiation successful
0812 11:10:00.218331 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":195}
I20260812 11:10:00.223323 28451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04: Bootstrap starting.
I20260812 11:10:00.225971 28451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:00.226837 28451 log.cc:824] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04: Log is configured to *not* fsync() on all Append() calls
I20260812 11:10:00.228902 28451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04: No bootstrap required, opened a new log
I20260812 11:10:00.233625 28451 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } }
I20260812 11:10:00.233929 28451 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:00.233968 28451 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57d679a7e23e4081b1f929452563ca04, State: Initialized, Role: FOLLOWER
I20260812 11:10:00.234640 28451 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } }
I20260812 11:10:00.235224 28461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } } }
I20260812 11:10:00.235472 28461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:10:00.236526 28451 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 11:10:00.243253 28478 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:10:00.243422 28478 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:10:00.247192 28235 master_runner.cc:428] Detected that this master 57d679a7e23e4081b1f929452563ca04 is joining an existing cluster
I20260812 11:10:00.247284 28235 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.44.252:33565
I20260812 11:10:00.249143 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 27475
I20260812 11:10:00.269179 28457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.252381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45044 (local address 127.17.44.253:35581)
0812 11:10:00.252605 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:00.252613 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:00.256859 (+ 4246us) server_negotiation.cc:407] Connection header received
0812 11:10:00.257135 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:00.257139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:00.257232 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:00.257296 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:00.258328 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.259258 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.260692 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.261105 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.261195 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.263443 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:00.263447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:00.263450 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:00.263480 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:00.266679 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.267389 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.267396 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.267401 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.267503 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.267855 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.267861 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.267865 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.268166 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:00.268258 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:00.268741 (+ 483us) server_negotiation.cc:299] Negotiation successful
0812 11:10:00.268927 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 11:10:00.269882 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.252341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:35581 (local address 127.0.0.1:45044)
0812 11:10:00.256702 (+ 4361us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:00.256722 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:00.256978 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:00.257529 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:00.257532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:00.257895 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:00.258155 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.258167 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.259422 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.259425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.260489 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.260523 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.261474 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.261478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.261583 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.262498 (+ 915us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:00.262528 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:00.263264 (+ 736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:00.266850 (+ 3586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.266857 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.266863 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.267239 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.267624 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.267630 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.267634 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.267733 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.268327 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:00.268333 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:00.268460 (+ 127us) client_negotiation.cc:769] Sending connection context
0812 11:10:00.269397 (+ 937us) client_negotiation.cc:240] Negotiation successful
0812 11:10:00.269585 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4221,"thread_start_us":1116,"threads_started":1}
I20260812 11:10:00.275243 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:36081
--local_ip_for_outbound_sockets=127.17.44.193
--tserver_master_addrs=127.17.44.254:33165
--webserver_port=33513
--webserver_interface=127.17.44.193
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.193.keytab
--principal=oryx/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.44.254:33165,127.17.44.253:35581
--tserver_master_addrs=127.17.44.254:33165,127.17.44.253:35581,127.17.44.252:33565 with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:10:00.280402 28455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.267476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34102 (local address 127.17.44.252:33565)
0812 11:10:00.271483 (+ 4007us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:00.271488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:00.271569 (+ 81us) server_negotiation.cc:407] Connection header received
0812 11:10:00.271711 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:00.271716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:00.271761 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:00.271809 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:00.272420 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.273221 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.274528 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.274850 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.274971 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.276371 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:00.276378 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:00.276383 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:00.276412 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:00.278251 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.278861 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.278868 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.278872 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.279006 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.279320 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.279326 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.279330 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.279570 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:00.279654 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:00.280023 (+ 369us) server_negotiation.cc:299] Negotiation successful
0812 11:10:00.280157 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3921}
I20260812 11:10:00.281411 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.267682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:33565 (local address 127.0.0.1:34102)
0812 11:10:00.271503 (+ 3821us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:00.271519 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:00.271623 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:00.271953 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:00.271958 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:00.272163 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:00.272299 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.272309 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.273470 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.273476 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.274406 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.274420 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.275117 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.275123 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.275227 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.275736 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:00.275757 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:00.276220 (+ 463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:00.278389 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.278392 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.278395 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.278656 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.279120 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.279125 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.279130 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.279226 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.279721 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:00.279726 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:00.279826 (+ 100us) client_negotiation.cc:769] Sending connection context
0812 11:10:00.280637 (+ 811us) client_negotiation.cc:240] Negotiation successful
0812 11:10:00.280794 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3773,"thread_start_us":161,"threads_started":1}
I20260812 11:10:00.286605 27442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.255122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34728 (local address 127.17.44.254:33165)
0812 11:10:00.255290 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:00.255297 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:00.268299 (+ 13002us) server_negotiation.cc:407] Connection header received
0812 11:10:00.268488 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:00.268491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:00.268548 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:00.268600 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:00.270162 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.271058 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.279582 (+ 8524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.279972 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.280061 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.282401 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:00.282406 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:00.282409 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:00.282443 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:00.284512 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.285106 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.285109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.285111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.285210 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.285569 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.285575 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.285577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.285835 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:00.285935 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:00.286323 (+ 388us) server_negotiation.cc:299] Negotiation successful
0812 11:10:00.286453 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 11:10:00.286761 28486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.255552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.0.0.1:34728)
0812 11:10:00.268196 (+ 12644us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:00.268211 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:00.268378 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:00.268754 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:00.268757 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:00.269014 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:00.269138 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.269146 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.278322 (+ 9176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.278326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.279373 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.279387 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.280270 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.280273 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.280393 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.281331 (+ 938us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:00.281355 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:00.282189 (+ 834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:00.284660 (+ 2471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.284667 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.284672 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.284993 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.285319 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.285326 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.285330 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.285457 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.285951 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:00.285957 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:00.286094 (+ 137us) client_negotiation.cc:769] Sending connection context
0812 11:10:00.286292 (+ 198us) client_negotiation.cc:240] Negotiation successful
0812 11:10:00.286487 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12560,"thread_start_us":714,"threads_started":1}
I20260812 11:10:00.297693 27878 heartbeater.cc:503] Master 127.17.44.254:33165 was elected leader, sending a full tablet report...
Aug 12 11:10:00 dist-test-slave-drl0 krb5kdc[27322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786532998, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.254@KRBTEST.COM for oryx/127.17.44.252@KRBTEST.COM
I20260812 11:10:00.313014 28461 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:10:00.313254 28461 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } }
I20260812 11:10:00.314316 28461 leader_election.cc:288] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fac979aa6d9a4662a500f789ee828ae3 (127.17.44.254:33165), 395bb06cf1e84d9ba73408995fc72721 (127.17.44.253:35581)
I20260812 11:10:00.314821 27399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "57d679a7e23e4081b1f929452563ca04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fac979aa6d9a4662a500f789ee828ae3" is_pre_election: true
I20260812 11:10:00.314960 28295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "57d679a7e23e4081b1f929452563ca04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "395bb06cf1e84d9ba73408995fc72721" is_pre_election: true
I20260812 11:10:00.315140 28295 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 1 LEARNER]: Handling vote request from an unknown peer 57d679a7e23e4081b1f929452563ca04
I20260812 11:10:00.315258 28295 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 57d679a7e23e4081b1f929452563ca04 for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260812 11:10:00.315618 28379 leader_election.cc:302] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57d679a7e23e4081b1f929452563ca04; no voters: 395bb06cf1e84d9ba73408995fc72721, fac979aa6d9a4662a500f789ee828ae3
I20260812 11:10:00.316534 28461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:00.317947 28461 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 11:10:00.320159 28222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.291466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:33565 (local address 127.0.0.1:34112)
0812 11:10:00.291597 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:00.291614 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:00.291702 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:00.292218 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:00.292223 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:00.292413 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:00.292522 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.292529 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.294053 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.294056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.295334 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.295346 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.296157 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.296161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.296255 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.296852 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:00.296868 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:00.298717 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:00.318157 (+ 19440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.318161 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.318163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.318448 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.318784 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.318787 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.318789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.319000 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.319523 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:00.319527 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:00.319640 (+ 113us) client_negotiation.cc:769] Sending connection context
0812 11:10:00.319777 (+ 137us) client_negotiation.cc:240] Negotiation successful
0812 11:10:00.319985 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 11:10:00.320189 28455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.291594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34112 (local address 127.17.44.252:33565)
0812 11:10:00.291881 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:00.291885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:00.291901 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:10:00.291968 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:00.291971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:00.292022 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:00.292104 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:00.292648 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.293428 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.295479 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.295920 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.296039 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.298906 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:00.298910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:00.298913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:00.298981 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:00.317966 (+ 18985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.318568 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.318572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.318574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.318671 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.319124 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.319128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.319130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.319385 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:00.319503 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:00.319903 (+ 400us) server_negotiation.cc:299] Negotiation successful
0812 11:10:00.320042 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195}
I20260812 11:10:00.321755 27384 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.44.252:33565: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "57d679a7e23e4081b1f929452563ca04"
member_type: NON_VOTER
last_known_addr {
host: "127.17.44.252"
port: 33565
}
attrs {
promote: true
}
}
dest_uuid: "fac979aa6d9a4662a500f789ee828ae3"
cas_config_opid_index: 5
I20260812 11:10:00.322398 27384 consensus_queue.cc:237] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: NON_VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: true } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: NON_VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: true } }
I20260812 11:10:00.323518 28225 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 57d679a7e23e4081b1f929452563ca04 (127.17.44.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: NON_VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: true } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: NON_VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: true } } }
I20260812 11:10:00.324978 28225 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.44.252:33565
I20260812 11:10:00.325160 28225 sys_catalog.cc:455] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "fac979aa6d9a4662a500f789ee828ae3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: NON_VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: true } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: NON_VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: true } } }
I20260812 11:10:00.325263 28225 sys_catalog.cc:458] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [sys.catalog]: This master's current role is: LEADER
I20260812 11:10:00.326323 28235 master.cc:537] Master@127.17.44.252:33565 shutting down...
I20260812 11:10:00.326640 28418 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 1 FOLLOWER]: Refusing update from remote peer fac979aa6d9a4662a500f789ee828ae3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 11:10:00.327243 28228 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: NON_VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 11:10:00.329597 27365 consensus_peers.cc:487] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 -> Peer 57d679a7e23e4081b1f929452563ca04 (127.17.44.252:33565): Couldn't send request to peer 57d679a7e23e4081b1f929452563ca04. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 11:10:00.334261 28235 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:10:00.334657 28235 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:10:00.334981 28235 tablet_replica.cc:330] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04: stopping tablet replica
I20260812 11:10:00.336278 28457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.325760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45048 (local address 127.17.44.253:35581)
0812 11:10:00.325915 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:00.325922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:00.326863 (+ 941us) server_negotiation.cc:407] Connection header received
0812 11:10:00.327106 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:00.327112 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:00.327189 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:00.327265 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:00.327673 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.328523 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.334281 (+ 5758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.335252 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.335364 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.335545 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:00.336045 (+ 500us) server_negotiation.cc:299] Negotiation successful
0812 11:10:00.336140 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 11:10:00.336727 28222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.325588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:35581 (local address 127.0.0.1:45048)
0812 11:10:00.326780 (+ 1192us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:00.326799 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:00.326978 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:00.327435 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:00.327438 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:00.327456 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:00.327545 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.327551 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.332023 (+ 4472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.332031 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.333842 (+ 1811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.333869 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.335676 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.335680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.335831 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.335904 (+ 73us) client_negotiation.cc:769] Sending connection context
0812 11:10:00.336494 (+ 590us) client_negotiation.cc:240] Negotiation successful
0812 11:10:00.336601 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1093}
I20260812 11:10:00.337440 28295 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 1 LEARNER]: Refusing update from remote peer fac979aa6d9a4662a500f789ee828ae3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 11:10:00.338493 28228 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: NON_VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 11:10:00.341042 28295 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 57d679a7e23e4081b1f929452563ca04 (127.17.44.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: NON_VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: true } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: NON_VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: true } } }
I20260812 11:10:00.344650 28495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fac979aa6d9a4662a500f789ee828ae3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: NON_VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: true } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: NON_VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: true } } }
I20260812 11:10:00.344871 28495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [sys.catalog]: This master's current role is: LEARNER
I20260812 11:10:00.345032 28336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fac979aa6d9a4662a500f789ee828ae3. Latest consensus state: current_term: 1 leader_uuid: "fac979aa6d9a4662a500f789ee828ae3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: NON_VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: true } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: NON_VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: true } } }
I20260812 11:10:00.345160 28336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [sys.catalog]: This master's current role is: LEARNER
I20260812 11:10:00.346470 28228 raft_consensus.cc:1050] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3: attempting to promote NON_VOTER 395bb06cf1e84d9ba73408995fc72721 to VOTER
I20260812 11:10:00.346993 28228 consensus_queue.cc:237] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: NON_VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: true } }
I20260812 11:10:00.348311 28222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.347955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:33565 (local address 127.0.0.1:34120)
0812 11:10:00.348137 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:00.348211 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260812 11:10:00.348429 28295 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 1 LEARNER]: Refusing update from remote peer fac979aa6d9a4662a500f789ee828ae3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 11:10:00.348780 28229 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 11:10:00.349733 28222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.349355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:33565 (local address 127.0.0.1:34124)
0812 11:10:00.349600 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:00.349659 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
W20260812 11:10:00.349929 27365 consensus_peers.cc:487] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 -> Peer 57d679a7e23e4081b1f929452563ca04 (127.17.44.252:33565): Couldn't send request to peer 57d679a7e23e4081b1f929452563ca04. Status: Network error: Client connection negotiation failed: client connection to 127.17.44.252:33565: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 11:10:00.351152 28228 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 395bb06cf1e84d9ba73408995fc72721 (127.17.44.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: NON_VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: true } } }
I20260812 11:10:00.354315 28293 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 395bb06cf1e84d9ba73408995fc72721 (127.17.44.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: NON_VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: true } } }
I20260812 11:10:00.357079 28336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fac979aa6d9a4662a500f789ee828ae3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: NON_VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: true } } }
I20260812 11:10:00.357235 28336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:10:00.358263 28235 master.cc:552] Master@127.17.44.252:33565 shutdown complete.
I20260812 11:10:00.358318 28235 master_runner.cc:305] Clearing existing system tablet
I20260812 11:10:00.358585 28235 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:00.358647 28235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:00.359170 28229 sys_catalog.cc:455] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "fac979aa6d9a4662a500f789ee828ae3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: NON_VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: true } } }
I20260812 11:10:00.359294 28229 sys_catalog.cc:458] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [sys.catalog]: This master's current role is: LEADER
I20260812 11:10:00.359489 28228 sys_catalog.cc:455] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fac979aa6d9a4662a500f789ee828ae3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: NON_VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: true } } }
I20260812 11:10:00.359579 28228 sys_catalog.cc:458] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [sys.catalog]: This master's current role is: LEADER
I20260812 11:10:00.360852 28235 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 11:10:00.361853 28498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:00.362076 28235 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:00.362200 28235 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 11:10:00.362232 28235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/wal
uuid: "57d679a7e23e4081b1f929452563ca04"
format_stamp: "Formatted at 2026-08-12 11:10:00 on dist-test-slave-drl0"
server_key: "7cc0a9b16deac62409ceab77c756f283"
server_key_iv: "328b26eb4599613f42b54a63638850c6"
server_key_version: "encryptionkey@0"
I20260812 11:10:00.364643 28235 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:10:00.367308 28235 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 11:10:00.367406 28235 log.cc:1196] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 11:10:00.367664 28235 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04: Deleting consensus metadata
I20260812 11:10:00.367866 28235 master_runner.cc:315] Copying system tablet from 127.17.44.254:33165
I20260812 11:10:00.369095 28235 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04: tablet copy: Beginning tablet copy session from remote peer at address 127.17.44.254:33165
I20260812 11:10:00.379233 27442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.369801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34742 (local address 127.17.44.254:33165)
0812 11:10:00.369957 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:00.369963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:00.370083 (+ 120us) server_negotiation.cc:407] Connection header received
0812 11:10:00.370205 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:00.370208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:00.370261 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:00.370362 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:00.370875 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.371796 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.373127 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.373452 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.373559 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.375254 (+ 1695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:00.375258 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:00.375261 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:00.375287 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:00.377234 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.377750 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.377754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.377756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.377855 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.378235 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.378239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.378241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.378469 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:00.378569 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:00.378901 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 11:10:00.379061 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 11:10:00.379320 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.369670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.0.0.1:34742)
0812 11:10:00.369982 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:00.369997 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:00.370117 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:00.370396 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:00.370399 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:00.370630 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:00.370741 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.370748 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.371943 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.371947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.372985 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.373000 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.373585 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.373589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.373681 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.374528 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:00.374548 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:00.375099 (+ 551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:00.377371 (+ 2272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.377374 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.377376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.377640 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.377991 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.377994 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.377996 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.378115 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.378582 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:00.378585 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:00.378690 (+ 105us) client_negotiation.cc:769] Sending connection context
0812 11:10:00.378826 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 11:10:00.379118 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":89,"threads_started":1}
I20260812 11:10:00.380236 27415 tablet_copy_service.cc:140] P fac979aa6d9a4662a500f789ee828ae3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 57d679a7e23e4081b1f929452563ca04 ({username='oryx', principal='oryx/127.17.44.252@KRBTEST.COM'} at 127.0.0.1:34742)
I20260812 11:10:00.380348 27415 tablet_copy_service.cc:161] P fac979aa6d9a4662a500f789ee828ae3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 57d679a7e23e4081b1f929452563ca04 at {username='oryx', principal='oryx/127.17.44.252@KRBTEST.COM'} at 127.0.0.1:34742: session id = 57d679a7e23e4081b1f929452563ca04-00000000000000000000000000000000
I20260812 11:10:00.381556 27415 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 11:10:00.382877 28235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 11:10:00.385720 28235 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04: tablet copy: Starting download of 0 data blocks...
I20260812 11:10:00.386083 28235 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04: tablet copy: Starting download of 1 WAL segments...
I20260812 11:10:00.387975 28235 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 11:10:00.389958 27415 tablet_copy_service.cc:342] P fac979aa6d9a4662a500f789ee828ae3: Request end of tablet copy session 57d679a7e23e4081b1f929452563ca04-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.44.252@KRBTEST.COM'} at 127.0.0.1:34742
I20260812 11:10:00.390054 27415 tablet_copy_service.cc:433] P fac979aa6d9a4662a500f789ee828ae3: ending tablet copy session 57d679a7e23e4081b1f929452563ca04-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 57d679a7e23e4081b1f929452563ca04
I20260812 11:10:00.399742 28235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:00.401775 28509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:00.408075 28510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:00.408072 28512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:00.408382 28235 server_base.cc:1034] running on GCE node
I20260812 11:10:00.408564 28235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:00.408874 28235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:00.410030 28235 hybrid_clock.cc:648] HybridClock initialized: now 1786533000410011 us; error 36 us; skew 500 ppm
Aug 12 11:10:00 dist-test-slave-drl0 krb5kdc[27322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533000, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:10:00.412622 28235 init.cc:377] Logged in from keytab as oryx/127.17.44.252@KRBTEST.COM (short username oryx)
I20260812 11:10:00.413116 28235 webserver.cc:460] Webserver started at http://127.17.44.252:37489/ using document root <none> and password file <none>
I20260812 11:10:00.413326 28235 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:00.413393 28235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:00.415021 28235 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 11:10:00.416183 28522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:00.416659 28235 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 11:10:00.416849 28235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/wal
uuid: "57d679a7e23e4081b1f929452563ca04"
format_stamp: "Formatted at 2026-08-12 11:10:00 on dist-test-slave-drl0"
server_key: "7cc0a9b16deac62409ceab77c756f283"
server_key_iv: "328b26eb4599613f42b54a63638850c6"
server_key_version: "encryptionkey@0"
I20260812 11:10:00.416944 28235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:00.436211 28235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:00.438426 28235 kserver.cc:163] Server-wide thread pool size limit: 3276
W20260812 11:10:00.438824 28492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:00.439180 28492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:00.439276 28492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 11:10:00.444862 28235 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.252:33565
I20260812 11:10:00.445288 28610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.252:33565 every 8 connection(s)
I20260812 11:10:00.445653 28235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/master-2/data/info.pb
W20260812 11:10:00.446327 28492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:00.446413 28492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:00.446517 28492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:10:00.447854 28611 sys_catalog.cc:263] Verifying existing consensus state
I20260812 11:10:00.449077 28611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04: Bootstrap starting.
I20260812 11:10:00.453224 28492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.193.keytab
--principal=oryx/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:36081
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=33513
--tserver_master_addrs=127.17.44.254:33165,127.17.44.253:35581,127.17.44.252:33565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:00.454447 28492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:00.455835 28492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 11:10:00.463465 28611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 11:10:00.463775 28611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04: Bootstrap complete.
I20260812 11:10:00.464330 28611 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: NON_VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: true } }
I20260812 11:10:00.464478 28611 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 57d679a7e23e4081b1f929452563ca04, State: Initialized, Role: LEARNER
I20260812 11:10:00.464608 28611 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: NON_VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: true } }
I20260812 11:10:00.465001 28621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: NON_VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: true } } }
I20260812 11:10:00.465090 28611 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:10:00.465216 28621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [sys.catalog]: This master's current role is: LEARNER
W20260812 11:10:00.467265 28617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:00.467265 28618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:00.467757 28492 server_base.cc:1034] running on GCE node
W20260812 11:10:00.467993 28620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:00.468475 28492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:00.468699 28640 catalog_manager.cc:1259] Loaded cluster ID: 7332444ac4b746989b256ca4915388fd
I20260812 11:10:00.468745 28640 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04: loading cluster ID for follower catalog manager: success
I20260812 11:10:00.469180 28492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:00.470451 28640 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04: acquiring CA information for follower catalog manager: success
I20260812 11:10:00.470472 28492 hybrid_clock.cc:648] HybridClock initialized: now 1786533000470417 us; error 45 us; skew 500 ppm
I20260812 11:10:00.471947 28640 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
Aug 12 11:10:00 dist-test-slave-drl0 krb5kdc[27322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533000, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:10:00.475334 28492 init.cc:377] Logged in from keytab as oryx/127.17.44.193@KRBTEST.COM (short username oryx)
I20260812 11:10:00.476691 28492 webserver.cc:460] Webserver started at http://127.17.44.193:33513/ using document root <none> and password file <none>
I20260812 11:10:00.477340 28492 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:00.477432 28492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:00.481947 28492 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:10:00.484814 28647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:00.486382 28492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 11:10:00.486524 28492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "9d36ca09efc643dd9d37dfedd9cd2403"
format_stamp: "Formatted at 2026-08-12 11:09:58 on dist-test-slave-drl0"
server_key: "eef279e2d24992fe4be72c3f046fb53a"
server_key_iv: "c2c99515999d71d65ecc8ce1f40f422f"
server_key_version: "encryptionkey@0"
I20260812 11:10:00.487437 28492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:00.507462 28492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:00.512120 28492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:00.512336 28492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:00.513044 28492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:10:00.514776 28492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:10:00.514870 28492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:00.514982 28492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:10:00.515017 28492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:00.529201 28492 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:36081
I20260812 11:10:00.529237 28816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:36081 every 8 connection(s)
I20260812 11:10:00.530422 28492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:10:00.535960 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 28492
I20260812 11:10:00.536252 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 27680
I20260812 11:10:00.536708 28080 heartbeater.cc:503] Master 127.17.44.254:33165 was elected leader, sending a full tablet report...
Aug 12 11:10:00 dist-test-slave-drl0 krb5kdc[27322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533000, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.253@KRBTEST.COM
Aug 12 11:10:00 dist-test-slave-drl0 krb5kdc[27322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533000, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM
Aug 12 11:10:00 dist-test-slave-drl0 krb5kdc[27322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533000, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.252@KRBTEST.COM
I20260812 11:10:00.551157 27442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.532451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:59833 (local address 127.17.44.254:33165)
0812 11:10:00.532583 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:00.532587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:00.533509 (+ 922us) server_negotiation.cc:407] Connection header received
0812 11:10:00.535243 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:00.535246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:00.535293 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:00.535399 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:00.537792 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.538692 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.541281 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.541650 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.541735 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.546006 (+ 4271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:00.546012 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:00.546017 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:00.546045 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:00.548425 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.549217 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.549220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.549223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.549311 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.549740 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.549743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.549745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.549975 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:00.550065 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:00.550819 (+ 754us) server_negotiation.cc:299] Negotiation successful
0812 11:10:00.550968 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 11:10:00.551270 28457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.532408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:40315 (local address 127.17.44.253:35581)
0812 11:10:00.532589 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:00.532595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:00.533506 (+ 911us) server_negotiation.cc:407] Connection header received
0812 11:10:00.535182 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:00.535188 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:00.535247 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:00.535345 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:00.537688 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.539244 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.541135 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.541424 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.541504 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.546552 (+ 5048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:00.546558 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:00.546564 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:00.546597 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:00.548816 (+ 2219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.549392 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.549399 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.549403 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.549490 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.549810 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.549816 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.549820 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.550062 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:00.550150 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:00.550846 (+ 696us) server_negotiation.cc:299] Negotiation successful
0812 11:10:00.551029 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 11:10:00.551844 28833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.534125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:41305 (local address 127.17.44.252:33565)
0812 11:10:00.534373 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:00.534377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:00.534613 (+ 236us) server_negotiation.cc:407] Connection header received
0812 11:10:00.535512 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:00.535515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:00.535572 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:00.535658 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:00.539948 (+ 4290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.541072 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.542493 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.542919 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.543163 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.546530 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:00.546536 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:00.546541 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:00.546571 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:00.548574 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.549210 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.549214 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.549215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.549307 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.549739 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.549743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.549745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.549975 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:00.550064 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:00.551538 (+ 1474us) server_negotiation.cc:299] Negotiation successful
0812 11:10:00.551667 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":84,"threads_started":1}
I20260812 11:10:00.552336 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:32917
--local_ip_for_outbound_sockets=127.17.44.194
--tserver_master_addrs=127.17.44.254:33165
--webserver_port=36571
--webserver_interface=127.17.44.194
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.194.keytab
--principal=oryx/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.44.254:33165,127.17.44.253:35581
--tserver_master_addrs=127.17.44.254:33165,127.17.44.253:35581,127.17.44.252:33565 with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:10:00.552579 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.532867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:35581 (local address 127.17.44.193:40315)
0812 11:10:00.533351 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:00.533404 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:00.534830 (+ 1426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:00.535571 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:00.535578 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:00.536267 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:00.537452 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.537485 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.539431 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.539438 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.540985 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.540997 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.541713 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.541811 (+ 98us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.542056 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.543049 (+ 993us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:00.543070 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:00.546388 (+ 3318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:00.548944 (+ 2556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.548950 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.548955 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.549267 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.549605 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.549610 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.549615 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.549709 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.550228 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:00.550234 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:00.550513 (+ 279us) client_negotiation.cc:769] Sending connection context
0812 11:10:00.551429 (+ 916us) client_negotiation.cc:240] Negotiation successful
0812 11:10:00.551596 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":32,"spinlock_wait_cycles":63616,"thread_start_us":201,"threads_started":1}
I20260812 11:10:00.553053 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.532867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.193:59833)
0812 11:10:00.533349 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:00.533405 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:00.534742 (+ 1337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:00.535568 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:00.535578 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:00.536229 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:00.537477 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.537511 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.538910 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.538916 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.540992 (+ 2076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.541007 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.541928 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.541932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.542060 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.542918 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:00.542974 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:00.545823 (+ 2849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:00.548574 (+ 2751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.548580 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.548598 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.549094 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.549417 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.549419 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.549421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.549624 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.550124 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:00.550131 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:00.550518 (+ 387us) client_negotiation.cc:769] Sending connection context
0812 11:10:00.551679 (+ 1161us) client_negotiation.cc:240] Negotiation successful
0812 11:10:00.551816 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":429,"spinlock_wait_cycles":19328,"thread_start_us":132,"threads_started":1}
I20260812 11:10:00.553512 28832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.533959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:33565 (local address 127.17.44.193:41305)
0812 11:10:00.534396 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:00.534410 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:00.534830 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:00.535806 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:00.535809 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:00.536268 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:00.539694 (+ 3426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.539708 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.541331 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.541335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.542332 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.542345 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.543311 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.543318 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.543437 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.543962 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:00.543982 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:00.546355 (+ 2373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:00.548683 (+ 2328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.548689 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.548692 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.549089 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.549416 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.549419 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.549421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.549624 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.550124 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:00.550131 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:00.550518 (+ 387us) client_negotiation.cc:769] Sending connection context
0812 11:10:00.550743 (+ 225us) client_negotiation.cc:240] Negotiation successful
0812 11:10:00.551127 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":3263,"spinlock_wait_cycles":15104,"thread_start_us":156,"threads_started":1}
I20260812 11:10:00.553957 28824 heartbeater.cc:347] Connected to a master server at 127.17.44.253:35581
I20260812 11:10:00.554154 28820 heartbeater.cc:347] Connected to a master server at 127.17.44.252:33565
I20260812 11:10:00.554231 28820 heartbeater.cc:464] Registering TS with master...
I20260812 11:10:00.554231 28824 heartbeater.cc:464] Registering TS with master...
I20260812 11:10:00.554749 28822 heartbeater.cc:347] Connected to a master server at 127.17.44.254:33165
I20260812 11:10:00.554849 28822 heartbeater.cc:464] Registering TS with master...
I20260812 11:10:00.554895 28824 heartbeater.cc:511] Master 127.17.44.253:35581 requested a full tablet report, sending...
I20260812 11:10:00.554895 28820 heartbeater.cc:511] Master 127.17.44.252:33565 requested a full tablet report, sending...
I20260812 11:10:00.555369 28822 heartbeater.cc:511] Master 127.17.44.254:33165 requested a full tablet report, sending...
I20260812 11:10:00.556599 28277 ts_manager.cc:194] Registered new tserver with Master: 9d36ca09efc643dd9d37dfedd9cd2403 (127.17.44.193:36081)
I20260812 11:10:00.557255 27384 ts_manager.cc:194] Re-registered known tserver with Master: 9d36ca09efc643dd9d37dfedd9cd2403 (127.17.44.193:36081)
I20260812 11:10:00.557776 28537 ts_manager.cc:194] Registered new tserver with Master: 9d36ca09efc643dd9d37dfedd9cd2403 (127.17.44.193:36081)
I20260812 11:10:00.557905 27384 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.44.193@KRBTEST.COM'} at 127.17.44.193:59833
W20260812 11:10:00.682864 28834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:00.683192 28834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:00.683236 28834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:00.687450 28834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:00.687506 28834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:00.687583 28834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:10:00.694590 28834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.194.keytab
--principal=oryx/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:32917
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=36571
--tserver_master_addrs=127.17.44.254:33165,127.17.44.253:35581,127.17.44.252:33565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:00.696036 28834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:00.697278 28834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:00.709564 28841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:00.709596 28845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:00.709597 28843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:00.709825 28834 server_base.cc:1034] running on GCE node
I20260812 11:10:00.710438 28834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:00.711166 28834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:00.712360 28834 hybrid_clock.cc:648] HybridClock initialized: now 1786533000712340 us; error 33 us; skew 500 ppm
Aug 12 11:10:00 dist-test-slave-drl0 krb5kdc[27322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533000, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:10:00.715339 28834 init.cc:377] Logged in from keytab as oryx/127.17.44.194@KRBTEST.COM (short username oryx)
I20260812 11:10:00.716295 28834 webserver.cc:460] Webserver started at http://127.17.44.194:36571/ using document root <none> and password file <none>
I20260812 11:10:00.716995 28834 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:00.717108 28834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:00.721469 28834 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:10:00.723842 28854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:00.725795 28834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 11:10:00.725955 28834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "084ef8e47f8e4b6d8eac7275ef273315"
format_stamp: "Formatted at 2026-08-12 11:09:59 on dist-test-slave-drl0"
server_key: "17d2918a53171f4fbb442f39df813428"
server_key_iv: "506bae71ecece50896ef514496f02b8c"
server_key_version: "encryptionkey@0"
I20260812 11:10:00.726523 28834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:00.766525 28834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:00.769917 28834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:00.770146 28834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:00.770776 28834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:10:00.772774 28834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:10:00.772886 28834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:00.772967 28834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:10:00.773154 28834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:00.784034 28834 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:32917
I20260812 11:10:00.784186 29022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:32917 every 8 connection(s)
I20260812 11:10:00.785162 28834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:10:00.785851 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 28834
I20260812 11:10:00.786027 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 27885
I20260812 11:10:00.805516 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:34889
--local_ip_for_outbound_sockets=127.17.44.195
--tserver_master_addrs=127.17.44.254:33165
--webserver_port=33575
--webserver_interface=127.17.44.195
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.195.keytab
--principal=oryx/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.44.254:33165,127.17.44.253:35581
--tserver_master_addrs=127.17.44.254:33165,127.17.44.253:35581,127.17.44.252:33565 with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 11:10:00 dist-test-slave-drl0 krb5kdc[27322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533000, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.252@KRBTEST.COM
Aug 12 11:10:00 dist-test-slave-drl0 krb5kdc[27322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533000, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.253@KRBTEST.COM
Aug 12 11:10:00 dist-test-slave-drl0 krb5kdc[27322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533000, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM
I20260812 11:10:00.814947 28833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.789553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:58435 (local address 127.17.44.252:33565)
0812 11:10:00.789661 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:00.789666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:00.796837 (+ 7171us) server_negotiation.cc:407] Connection header received
0812 11:10:00.798448 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:00.798451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:00.798507 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:00.798638 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:00.800694 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.801535 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.804023 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.804392 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.804471 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.809612 (+ 5141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:00.809619 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:00.809625 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:00.809662 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:00.811755 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.812619 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.812624 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.812628 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.812722 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.813325 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.813330 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.813333 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.813636 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:00.813728 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:00.814569 (+ 841us) server_negotiation.cc:299] Negotiation successful
0812 11:10:00.814715 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 11:10:00.815487 28457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.793591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:39949 (local address 127.17.44.253:35581)
0812 11:10:00.796059 (+ 2468us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:00.796062 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:00.799221 (+ 3159us) server_negotiation.cc:407] Connection header received
0812 11:10:00.799418 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:00.799421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:00.799484 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:00.799544 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:00.800694 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.801535 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.804023 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.804388 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.804475 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.808867 (+ 4392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:00.808871 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:00.808874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:00.808902 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:00.810796 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.812636 (+ 1840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.812642 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.812646 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.812768 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.813349 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.813356 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.813361 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.813672 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:00.813811 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:00.815087 (+ 1276us) server_negotiation.cc:299] Negotiation successful
0812 11:10:00.815237 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2384}
I20260812 11:10:00.815656 27442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.787432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:46665 (local address 127.17.44.254:33165)
0812 11:10:00.787608 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:00.787616 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:00.798786 (+ 11170us) server_negotiation.cc:407] Connection header received
0812 11:10:00.798956 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:00.798962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:00.799025 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:00.799083 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:00.800703 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.801605 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.804120 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.804480 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.804582 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.811481 (+ 6899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:00.811484 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:00.811486 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:00.811516 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:00.813374 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.814054 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.814058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.814060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.814162 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:00.814519 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:00.814523 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:00.814524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:00.814762 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:00.814850 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:00.815356 (+ 506us) server_negotiation.cc:299] Negotiation successful
0812 11:10:00.815481 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 11:10:00.817404 29031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.787707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.194:46665)
0812 11:10:00.798668 (+ 10961us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:00.798692 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:00.798812 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:00.799259 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:00.799265 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:00.799611 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:00.800515 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.800540 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.802031 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.802038 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.803896 (+ 1858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.803920 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.804842 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.804848 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.807871 (+ 3023us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.808896 (+ 1025us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:00.808913 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:00.811222 (+ 2309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:00.813528 (+ 2306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.813531 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.813533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.813821 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.814283 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.814286 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.814288 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.814387 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.814913 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:00.814916 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:00.815136 (+ 220us) client_negotiation.cc:769] Sending connection context
0812 11:10:00.815859 (+ 723us) client_negotiation.cc:240] Negotiation successful
0812 11:10:00.816000 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10869,"mutex_wait_us":284,"spinlock_wait_cycles":54528,"thread_start_us":1037,"threads_started":1}
I20260812 11:10:00.817404 29033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.787655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:33565 (local address 127.17.44.194:58435)
0812 11:10:00.796609 (+ 8954us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:00.796682 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:00.798139 (+ 1457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:00.798870 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:00.798883 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:00.799600 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:00.800515 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.800537 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.801750 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.801753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.803876 (+ 2123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.803883 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.804748 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.804752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.805050 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.806012 (+ 962us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:00.806036 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:00.808655 (+ 2619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:00.811898 (+ 3243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.811906 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.811924 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.812480 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.812854 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.812858 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.812862 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.813192 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.813806 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:00.813812 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:00.814252 (+ 440us) client_negotiation.cc:769] Sending connection context
0812 11:10:00.815339 (+ 1087us) client_negotiation.cc:240] Negotiation successful
0812 11:10:00.815492 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8688,"mutex_wait_us":606,"spinlock_wait_cycles":35712,"thread_start_us":1035,"threads_started":1}
I20260812 11:10:00.817449 29034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.791616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:35581 (local address 127.17.44.194:39949)
0812 11:10:00.799118 (+ 7502us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:00.799134 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:00.799294 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:00.799700 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:00.799703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:00.799879 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:00.800515 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.800537 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.801735 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.801738 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.803868 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.803883 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.804748 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.804752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.805044 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.806725 (+ 1681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:00.806750 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:00.808682 (+ 1932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:00.811921 (+ 3239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.811928 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.811934 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.812486 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.812903 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:00.812911 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:00.812916 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:00.813205 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:00.813872 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:00.813878 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:00.814257 (+ 379us) client_negotiation.cc:769] Sending connection context
0812 11:10:00.814496 (+ 239us) client_negotiation.cc:240] Negotiation successful
0812 11:10:00.815742 (+ 1246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7409,"mutex_wait_us":402,"spinlock_wait_cycles":27264,"thread_start_us":1663,"threads_started":1}
I20260812 11:10:00.819319 29024 heartbeater.cc:347] Connected to a master server at 127.17.44.252:33565
I20260812 11:10:00.819590 29024 heartbeater.cc:464] Registering TS with master...
I20260812 11:10:00.819887 29025 heartbeater.cc:347] Connected to a master server at 127.17.44.254:33165
I20260812 11:10:00.820047 29025 heartbeater.cc:464] Registering TS with master...
I20260812 11:10:00.820158 29029 heartbeater.cc:347] Connected to a master server at 127.17.44.253:35581
I20260812 11:10:00.820222 29029 heartbeater.cc:464] Registering TS with master...
I20260812 11:10:00.820293 29024 heartbeater.cc:511] Master 127.17.44.252:33565 requested a full tablet report, sending...
I20260812 11:10:00.820327 29025 heartbeater.cc:511] Master 127.17.44.254:33165 requested a full tablet report, sending...
I20260812 11:10:00.820338 29029 heartbeater.cc:511] Master 127.17.44.253:35581 requested a full tablet report, sending...
I20260812 11:10:00.821868 27384 ts_manager.cc:194] Re-registered known tserver with Master: 084ef8e47f8e4b6d8eac7275ef273315 (127.17.44.194:32917)
I20260812 11:10:00.821974 28277 ts_manager.cc:194] Registered new tserver with Master: 084ef8e47f8e4b6d8eac7275ef273315 (127.17.44.194:32917)
I20260812 11:10:00.821993 28537 ts_manager.cc:194] Registered new tserver with Master: 084ef8e47f8e4b6d8eac7275ef273315 (127.17.44.194:32917)
I20260812 11:10:00.822908 27384 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.44.194@KRBTEST.COM'} at 127.17.44.194:46665
I20260812 11:10:00.926331 29037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.921830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:33565 (local address 127.0.0.1:34132)
0812 11:10:00.922229 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:00.922242 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:00.922385 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:00.922750 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:00.922753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:00.922773 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:00.922874 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.922879 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.924047 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.924050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.924846 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:00.924855 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.925964 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.925967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:00.926039 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.926062 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 11:10:00.926140 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 11:10:00.926202 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":112,"threads_started":1}
I20260812 11:10:00.926384 28833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:00.921897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34132 (local address 127.17.44.252:33565)
0812 11:10:00.922062 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:00.922068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:00.922357 (+ 289us) server_negotiation.cc:407] Connection header received
0812 11:10:00.922492 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:00.922495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:00.922567 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:00.922691 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:00.923059 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.923913 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.925057 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:00.925793 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:00.925925 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:00.926083 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:00.926157 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 11:10:00.926240 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 11:10:00.927093 28554 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260812 11:10:00.927524 28621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fac979aa6d9a4662a500f789ee828ae3. Latest consensus state: current_term: 1 leader_uuid: "fac979aa6d9a4662a500f789ee828ae3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: NON_VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: true } } }
I20260812 11:10:00.930099 28621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [sys.catalog]: This master's current role is: LEARNER
W20260812 11:10:00.939152 29035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:00.939428 29035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:00.939514 29035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:00.943602 29035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:00.943681 29035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:00.943778 29035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:10:00.949339 29035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.195.keytab
--principal=oryx/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:34889
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=33575
--tserver_master_addrs=127.17.44.254:33165,127.17.44.253:35581,127.17.44.252:33565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:00.950551 29035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:00.951689 29035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:00.962857 29051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:00.962882 29046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:00.962882 29047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:00.963096 29035 server_base.cc:1034] running on GCE node
I20260812 11:10:00.963550 29035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:00.964138 29035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:00.965314 29035 hybrid_clock.cc:648] HybridClock initialized: now 1786533000965291 us; error 32 us; skew 500 ppm
Aug 12 11:10:00 dist-test-slave-drl0 krb5kdc[27322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533000, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:10:00.968685 29035 init.cc:377] Logged in from keytab as oryx/127.17.44.195@KRBTEST.COM (short username oryx)
I20260812 11:10:00.969574 29035 webserver.cc:460] Webserver started at http://127.17.44.195:33575/ using document root <none> and password file <none>
I20260812 11:10:00.970142 29035 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:00.970260 29035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:00.974169 29035 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:10:00.976450 29063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:00.977841 29035 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 11:10:00.977957 29035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "ceb2d2ea9c474b06853d1891161296a8"
format_stamp: "Formatted at 2026-08-12 11:09:59 on dist-test-slave-drl0"
server_key: "f1a6a49c7584b580cbd698511375d5f1"
server_key_iv: "fcd15d65de86494432dec068c5dd658d"
server_key_version: "encryptionkey@0"
I20260812 11:10:00.978466 29035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:00.995841 29035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:00.999653 29035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:00.999811 29035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:01.000455 29035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:10:01.001380 29035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:10:01.001430 29035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:01.001472 29035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:10:01.001488 29035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:01.012562 29035 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:34889
I20260812 11:10:01.012588 29228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:34889 every 8 connection(s)
I20260812 11:10:01.013584 29035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:10:01.016345 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 29035
Aug 12 11:10:01 dist-test-slave-drl0 krb5kdc[27322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533000, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM
Aug 12 11:10:01 dist-test-slave-drl0 krb5kdc[27322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533000, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.253@KRBTEST.COM
I20260812 11:10:01.032368 29243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:01.018746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45062 (local address 127.17.44.253:35581)
0812 11:10:01.019435 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:01.019440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:01.019714 (+ 274us) server_negotiation.cc:407] Connection header received
0812 11:10:01.019930 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:01.019934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:01.019990 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:01.020094 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:01.021250 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.022044 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.023327 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.023761 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.023852 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:01.025748 (+ 1896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:01.025752 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:01.025755 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:01.025782 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:01.028365 (+ 2583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:01.030398 (+ 2033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:01.030401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:01.030403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:01.030498 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:01.030828 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:01.030832 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:01.030835 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:01.031097 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:01.031236 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:01.032019 (+ 783us) server_negotiation.cc:299] Negotiation successful
0812 11:10:01.032151 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":84,"threads_started":1}
I20260812 11:10:01.032853 29249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:01.021630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34146 (local address 127.17.44.252:33565)
0812 11:10:01.022145 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:01.022149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:01.022691 (+ 542us) server_negotiation.cc:407] Connection header received
0812 11:10:01.022847 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:01.022850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:01.022906 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:01.022982 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:01.023913 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.024674 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.025894 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.026289 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.026369 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:01.028712 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:01.028716 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:01.028718 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:01.028743 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:01.030815 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:01.031401 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:01.031404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:01.031406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:01.031499 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:01.031867 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:01.031871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:01.031873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:01.032093 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:01.032196 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:01.032553 (+ 357us) server_negotiation.cc:299] Negotiation successful
0812 11:10:01.032683 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":81,"threads_started":1}
Aug 12 11:10:01 dist-test-slave-drl0 krb5kdc[27322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533000, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.252@KRBTEST.COM
I20260812 11:10:01.036841 27442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:01.017042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:50385 (local address 127.17.44.254:33165)
0812 11:10:01.017254 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:01.017259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:01.017488 (+ 229us) server_negotiation.cc:407] Connection header received
0812 11:10:01.019888 (+ 2400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:01.019895 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:01.019965 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:01.020050 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:01.022532 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.023452 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.025398 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.025722 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.025804 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:01.030411 (+ 4607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:01.030415 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:01.030417 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:01.030443 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:01.032735 (+ 2292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:01.034140 (+ 1405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:01.034144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:01.034146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:01.034234 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:01.035061 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:01.035067 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:01.035074 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:01.035342 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:01.035445 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:01.036546 (+ 1101us) server_negotiation.cc:299] Negotiation successful
0812 11:10:01.036669 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260812 11:10:01.037827 29233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:01.016023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.195:50385)
0812 11:10:01.017347 (+ 1324us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:01.017389 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:01.019431 (+ 2042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:01.020454 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:01.020464 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:01.021193 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:01.022356 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:01.022380 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.023633 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.023635 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:01.025249 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:01.025260 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.026031 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.026037 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:01.026283 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:01.027285 (+ 1002us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:01.027313 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:01.030141 (+ 2828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:01.033647 (+ 3506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:01.033653 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:01.033671 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:01.034043 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:01.034699 (+ 656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:01.034704 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:01.034707 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:01.034908 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:01.035513 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:01.035521 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:01.035872 (+ 351us) client_negotiation.cc:769] Sending connection context
0812 11:10:01.036100 (+ 228us) client_negotiation.cc:240] Negotiation successful
0812 11:10:01.036655 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1243,"mutex_wait_us":135,"spinlock_wait_cycles":964480,"thread_start_us":178,"threads_started":1}
I20260812 11:10:01.038818 28457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:01.016211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:47235 (local address 127.17.44.253:35581)
0812 11:10:01.016347 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:01.016351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:01.017488 (+ 1137us) server_negotiation.cc:407] Connection header received
0812 11:10:01.020877 (+ 3389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:01.020884 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:01.020939 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:01.021019 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:01.022531 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.023460 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.025533 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.025846 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.025921 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:01.034404 (+ 8483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:01.034407 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:01.034410 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:01.034435 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:01.036589 (+ 2154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:01.037172 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:01.037179 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:01.037181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:01.037271 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:01.037603 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:01.037608 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:01.037610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:01.037821 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:01.037879 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:01.038554 (+ 675us) server_negotiation.cc:299] Negotiation successful
0812 11:10:01.038667 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 11:10:01.039212 29236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:01.016023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:35581 (local address 127.17.44.195:47235)
0812 11:10:01.017331 (+ 1308us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:01.017388 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:01.020715 (+ 3327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:01.021178 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:01.021181 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:01.021434 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:01.022362 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:01.022380 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.023631 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.023634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:01.025403 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:01.025412 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.026052 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.026054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:01.026515 (+ 461us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:01.027909 (+ 1394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:01.027924 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:01.032524 (+ 4600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:01.036713 (+ 4189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:01.036717 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:01.036719 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:01.037062 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:01.037365 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:01.037367 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:01.037368 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:01.037489 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:01.037991 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:01.037995 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:01.038379 (+ 384us) client_negotiation.cc:769] Sending connection context
0812 11:10:01.038499 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 11:10:01.038641 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1073,"mutex_wait_us":521,"spinlock_wait_cycles":6400,"thread_start_us":135,"threads_started":1}
I20260812 11:10:01.039269 29230 heartbeater.cc:347] Connected to a master server at 127.17.44.254:33165
I20260812 11:10:01.039532 29230 heartbeater.cc:464] Registering TS with master...
I20260812 11:10:01.039831 29231 heartbeater.cc:347] Connected to a master server at 127.17.44.253:35581
I20260812 11:10:01.039925 29231 heartbeater.cc:464] Registering TS with master...
I20260812 11:10:01.040141 29231 heartbeater.cc:511] Master 127.17.44.253:35581 requested a full tablet report, sending...
I20260812 11:10:01.040144 29230 heartbeater.cc:511] Master 127.17.44.254:33165 requested a full tablet report, sending...
I20260812 11:10:01.040611 29235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:01.016043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:33565 (local address 127.17.44.195:38491)
0812 11:10:01.017687 (+ 1644us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:01.017702 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:01.019913 (+ 2211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:01.020454 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:01.020464 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:01.026753 (+ 6289us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:01.027110 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:01.027120 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.030031 (+ 2911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.030035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:01.031085 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:01.031104 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.031930 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.031934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:01.032037 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:01.032689 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:01.032704 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:01.034136 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:01.036241 (+ 2105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:01.036244 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:01.036246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:01.036518 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:01.039179 (+ 2661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:01.039187 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:01.039188 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:01.039300 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:01.040098 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:01.040101 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:01.040179 (+ 78us) client_negotiation.cc:769] Sending connection context
0812 11:10:01.040319 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 11:10:01.040457 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1561,"mutex_wait_us":320,"thread_start_us":144,"threads_started":1}
I20260812 11:10:01.041090 28833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:01.017258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:38491 (local address 127.17.44.252:33565)
0812 11:10:01.017471 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:01.017476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:01.017779 (+ 303us) server_negotiation.cc:407] Connection header received
0812 11:10:01.020051 (+ 2272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:01.020054 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:01.020125 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:01.020189 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:01.028605 (+ 8416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.029639 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.031244 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.031642 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.031730 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:01.034390 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:01.034394 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:01.034396 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:01.034420 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:01.036117 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:01.037663 (+ 1546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:01.037667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:01.037668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:01.037754 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:01.039599 (+ 1845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:01.039603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:01.039605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:01.039951 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:01.040037 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:01.040405 (+ 368us) server_negotiation.cc:299] Negotiation successful
0812 11:10:01.040525 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260812 11:10:01.041306 27384 ts_manager.cc:194] Re-registered known tserver with Master: ceb2d2ea9c474b06853d1891161296a8 (127.17.44.195:34889)
I20260812 11:10:01.041360 28277 ts_manager.cc:194] Registered new tserver with Master: ceb2d2ea9c474b06853d1891161296a8 (127.17.44.195:34889)
I20260812 11:10:01.041754 29229 heartbeater.cc:347] Connected to a master server at 127.17.44.252:33565
I20260812 11:10:01.041836 29229 heartbeater.cc:464] Registering TS with master...
I20260812 11:10:01.041996 29229 heartbeater.cc:511] Master 127.17.44.252:33565 requested a full tablet report, sending...
I20260812 11:10:01.042106 27384 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.44.195@KRBTEST.COM'} at 127.17.44.195:50385
I20260812 11:10:01.042882 28537 ts_manager.cc:194] Registered new tserver with Master: ceb2d2ea9c474b06853d1891161296a8 (127.17.44.195:34889)
I20260812 11:10:01.047691 29251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:01.021870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34752 (local address 127.17.44.254:33165)
0812 11:10:01.028738 (+ 6868us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:01.028742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:01.028755 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:10:01.028790 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:01.028793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:01.028838 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:01.034268 (+ 5430us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:01.034389 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.035377 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.036655 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.036975 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.037460 (+ 485us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:01.040900 (+ 3440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:01.042334 (+ 1434us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:01.042337 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:01.042353 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:01.044398 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:01.044960 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:01.044962 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:01.044964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:01.045059 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:01.045642 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:01.045645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:01.045646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:01.046379 (+ 733us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:01.046693 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:01.047111 (+ 418us) server_negotiation.cc:299] Negotiation successful
0812 11:10:01.047370 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6791,"thread_start_us":81,"threads_started":1}
I20260812 11:10:01.051450 27384 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:10:01.054528 27384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:10:01.057760 29036 sys_catalog.cc:455] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fac979aa6d9a4662a500f789ee828ae3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: NON_VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: true } } }
I20260812 11:10:01.057981 29036 sys_catalog.cc:458] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [sys.catalog]: This master's current role is: LEADER
I20260812 11:10:01.062024 29255 raft_consensus.cc:1050] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3: attempting to promote NON_VOTER 57d679a7e23e4081b1f929452563ca04 to VOTER
I20260812 11:10:01.062439 29255 consensus_queue.cc:237] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: false } }
I20260812 11:10:01.064183 28293 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 1 FOLLOWER]: Refusing update from remote peer fac979aa6d9a4662a500f789ee828ae3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 11:10:01.064540 28554 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 1 LEARNER]: Refusing update from remote peer fac979aa6d9a4662a500f789ee828ae3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 11:10:01.064604 29253 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 11:10:01.065024 29253 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 11:10:01.067175 29253 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 57d679a7e23e4081b1f929452563ca04 (127.17.44.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: false } } }
I20260812 11:10:01.067508 28293 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 57d679a7e23e4081b1f929452563ca04 (127.17.44.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: false } } }
I20260812 11:10:01.067706 28554 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 57d679a7e23e4081b1f929452563ca04 (127.17.44.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: false } } }
I20260812 11:10:01.068811 29253 sys_catalog.cc:455] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "fac979aa6d9a4662a500f789ee828ae3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: false } } }
I20260812 11:10:01.068912 29253 sys_catalog.cc:458] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [sys.catalog]: This master's current role is: LEADER
I20260812 11:10:01.069031 29253 sys_catalog.cc:455] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fac979aa6d9a4662a500f789ee828ae3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: false } } }
I20260812 11:10:01.069105 29253 sys_catalog.cc:458] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [sys.catalog]: This master's current role is: LEADER
I20260812 11:10:01.069221 29253 sys_catalog.cc:455] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fac979aa6d9a4662a500f789ee828ae3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: false } } }
I20260812 11:10:01.069350 29253 sys_catalog.cc:458] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3 [sys.catalog]: This master's current role is: LEADER
I20260812 11:10:01.069826 28621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fac979aa6d9a4662a500f789ee828ae3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: false } } }
I20260812 11:10:01.069934 28621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:10:01.070179 29272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fac979aa6d9a4662a500f789ee828ae3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: false } } }
I20260812 11:10:01.070561 29272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:10:01.078416 29037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:01.072096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:32917 (local address 127.0.0.1:54196)
0812 11:10:01.072773 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:01.072792 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:01.072985 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:01.074192 (+ 1207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:01.074195 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:01.074212 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:01.074334 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:01.074341 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.075958 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.075962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:01.076865 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:01.076872 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.077994 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.077998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:01.078090 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:01.078113 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 11:10:01.078181 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 11:10:01.078277 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585}
I20260812 11:10:01.078464 29277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:01.073249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54196 (local address 127.17.44.194:32917)
0812 11:10:01.073602 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:01.073611 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:01.073632 (+ 21us) server_negotiation.cc:407] Connection header received
0812 11:10:01.073700 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:01.073704 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:01.074007 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:01.074515 (+ 508us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:01.074693 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.075838 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.076991 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.077864 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.077986 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:01.078154 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:01.078212 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 11:10:01.078307 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":189,"threads_started":1}
I20260812 11:10:01.081393 29274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:01.073939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:34889 (local address 127.0.0.1:49244)
0812 11:10:01.075374 (+ 1435us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:01.075386 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:01.075472 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:01.076589 (+ 1117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:01.076592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:01.076614 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:01.076705 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:01.076711 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.078217 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.078220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:01.079667 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:01.079689 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.080967 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.080971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:01.081071 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:01.081098 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 11:10:01.081170 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 11:10:01.081250 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1370,"mutex_wait_us":50,"thread_start_us":121,"threads_started":1}
I20260812 11:10:01.081853 29280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:01.074424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49244 (local address 127.17.44.195:34889)
0812 11:10:01.075913 (+ 1489us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:01.075932 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:01.075967 (+ 35us) server_negotiation.cc:407] Connection header received
0812 11:10:01.076088 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:01.076095 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:01.076379 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:01.076598 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:01.076864 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.078071 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.079874 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.080792 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.080927 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:01.081263 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:01.081340 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 11:10:01.081474 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1332,"thread_start_us":196,"threads_started":1}
I20260812 11:10:01.082207 28923 tablet_service.cc:1467] Processing CreateTablet for tablet 67e9a68c52554703ab56fada9459de86 (DEFAULT_TABLE table=test-table [id=31111d04c953412d8295119e0eb3f1db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:10:01.083485 28923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67e9a68c52554703ab56fada9459de86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:01.085506 29132 tablet_service.cc:1467] Processing CreateTablet for tablet 67e9a68c52554703ab56fada9459de86 (DEFAULT_TABLE table=test-table [id=31111d04c953412d8295119e0eb3f1db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:10:01.086753 29132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67e9a68c52554703ab56fada9459de86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:01.090853 29279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:01.072688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:36081 (local address 127.0.0.1:42798)
0812 11:10:01.074799 (+ 2111us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:01.074821 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:01.075041 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:01.075550 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:01.075554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:01.075658 (+ 104us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:01.075824 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:01.075835 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.078132 (+ 2297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.078139 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:01.079710 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:01.079722 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.082597 (+ 2875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.082601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:01.082713 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:01.082741 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 11:10:01.090547 (+ 7806us) client_negotiation.cc:240] Negotiation successful
0812 11:10:01.090687 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1439,"mutex_wait_us":60,"thread_start_us":109,"threads_started":1}
I20260812 11:10:01.091684 29276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:01.072989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42798 (local address 127.17.44.193:36081)
0812 11:10:01.073287 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:01.073295 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:01.074992 (+ 1697us) server_negotiation.cc:407] Connection header received
0812 11:10:01.075198 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:01.075204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:01.075383 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:01.075545 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:01.076043 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.077954 (+ 1911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.079889 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.082361 (+ 2472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.082563 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:01.082868 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:01.091393 (+ 8525us) server_negotiation.cc:299] Negotiation successful
0812 11:10:01.091550 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":83,"threads_started":1}
I20260812 11:10:01.091854 29282 tablet_bootstrap.cc:492] T 67e9a68c52554703ab56fada9459de86 P 084ef8e47f8e4b6d8eac7275ef273315: Bootstrap starting.
I20260812 11:10:01.095081 28716 tablet_service.cc:1467] Processing CreateTablet for tablet 67e9a68c52554703ab56fada9459de86 (DEFAULT_TABLE table=test-table [id=31111d04c953412d8295119e0eb3f1db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:10:01.095427 29282 tablet_bootstrap.cc:654] T 67e9a68c52554703ab56fada9459de86 P 084ef8e47f8e4b6d8eac7275ef273315: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:01.096115 28716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67e9a68c52554703ab56fada9459de86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:01.096896 29282 log.cc:824] T 67e9a68c52554703ab56fada9459de86 P 084ef8e47f8e4b6d8eac7275ef273315: Log is configured to *not* fsync() on all Append() calls
I20260812 11:10:01.097755 29283 tablet_bootstrap.cc:492] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8: Bootstrap starting.
I20260812 11:10:01.099861 29282 tablet_bootstrap.cc:492] T 67e9a68c52554703ab56fada9459de86 P 084ef8e47f8e4b6d8eac7275ef273315: No bootstrap required, opened a new log
I20260812 11:10:01.100162 29282 ts_tablet_manager.cc:1397] T 67e9a68c52554703ab56fada9459de86 P 084ef8e47f8e4b6d8eac7275ef273315: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260812 11:10:01.101181 29283 tablet_bootstrap.cc:654] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:01.102839 29283 log.cc:824] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8: Log is configured to *not* fsync() on all Append() calls
I20260812 11:10:01.104122 29285 tablet_bootstrap.cc:492] T 67e9a68c52554703ab56fada9459de86 P 9d36ca09efc643dd9d37dfedd9cd2403: Bootstrap starting.
I20260812 11:10:01.106361 29283 tablet_bootstrap.cc:492] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8: No bootstrap required, opened a new log
I20260812 11:10:01.106629 29283 ts_tablet_manager.cc:1397] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260812 11:10:01.107790 29285 tablet_bootstrap.cc:654] T 67e9a68c52554703ab56fada9459de86 P 9d36ca09efc643dd9d37dfedd9cd2403: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:01.107461 29282 raft_consensus.cc:348] T 67e9a68c52554703ab56fada9459de86 P 084ef8e47f8e4b6d8eac7275ef273315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "084ef8e47f8e4b6d8eac7275ef273315" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 32917 } } peers { permanent_uuid: "ceb2d2ea9c474b06853d1891161296a8" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34889 } } peers { permanent_uuid: "9d36ca09efc643dd9d37dfedd9cd2403" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36081 } }
I20260812 11:10:01.108024 29282 raft_consensus.cc:374] T 67e9a68c52554703ab56fada9459de86 P 084ef8e47f8e4b6d8eac7275ef273315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:01.108114 29282 raft_consensus.cc:729] T 67e9a68c52554703ab56fada9459de86 P 084ef8e47f8e4b6d8eac7275ef273315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 084ef8e47f8e4b6d8eac7275ef273315, State: Initialized, Role: FOLLOWER
I20260812 11:10:01.108734 29282 consensus_queue.cc:260] T 67e9a68c52554703ab56fada9459de86 P 084ef8e47f8e4b6d8eac7275ef273315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "084ef8e47f8e4b6d8eac7275ef273315" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 32917 } } peers { permanent_uuid: "ceb2d2ea9c474b06853d1891161296a8" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34889 } } peers { permanent_uuid: "9d36ca09efc643dd9d37dfedd9cd2403" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36081 } }
I20260812 11:10:01.108880 29285 log.cc:824] T 67e9a68c52554703ab56fada9459de86 P 9d36ca09efc643dd9d37dfedd9cd2403: Log is configured to *not* fsync() on all Append() calls
I20260812 11:10:01.110028 29282 ts_tablet_manager.cc:1428] T 67e9a68c52554703ab56fada9459de86 P 084ef8e47f8e4b6d8eac7275ef273315: Time spent starting tablet: real 0.010s user 0.006s sys 0.004s
I20260812 11:10:01.110260 29025 heartbeater.cc:503] Master 127.17.44.254:33165 was elected leader, sending a full tablet report...
I20260812 11:10:01.111048 29285 tablet_bootstrap.cc:492] T 67e9a68c52554703ab56fada9459de86 P 9d36ca09efc643dd9d37dfedd9cd2403: No bootstrap required, opened a new log
I20260812 11:10:01.111249 29285 ts_tablet_manager.cc:1397] T 67e9a68c52554703ab56fada9459de86 P 9d36ca09efc643dd9d37dfedd9cd2403: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260812 11:10:01.112865 29283 raft_consensus.cc:348] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "084ef8e47f8e4b6d8eac7275ef273315" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 32917 } } peers { permanent_uuid: "ceb2d2ea9c474b06853d1891161296a8" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34889 } } peers { permanent_uuid: "9d36ca09efc643dd9d37dfedd9cd2403" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36081 } }
I20260812 11:10:01.113066 29283 raft_consensus.cc:374] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:01.113096 29283 raft_consensus.cc:729] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb2d2ea9c474b06853d1891161296a8, State: Initialized, Role: FOLLOWER
I20260812 11:10:01.113539 29283 consensus_queue.cc:260] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "084ef8e47f8e4b6d8eac7275ef273315" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 32917 } } peers { permanent_uuid: "ceb2d2ea9c474b06853d1891161296a8" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34889 } } peers { permanent_uuid: "9d36ca09efc643dd9d37dfedd9cd2403" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36081 } }
I20260812 11:10:01.114228 29230 heartbeater.cc:503] Master 127.17.44.254:33165 was elected leader, sending a full tablet report...
I20260812 11:10:01.114547 29283 ts_tablet_manager.cc:1428] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8: Time spent starting tablet: real 0.008s user 0.008s sys 0.001s
I20260812 11:10:01.116600 29285 raft_consensus.cc:348] T 67e9a68c52554703ab56fada9459de86 P 9d36ca09efc643dd9d37dfedd9cd2403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "084ef8e47f8e4b6d8eac7275ef273315" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 32917 } } peers { permanent_uuid: "ceb2d2ea9c474b06853d1891161296a8" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34889 } } peers { permanent_uuid: "9d36ca09efc643dd9d37dfedd9cd2403" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36081 } }
I20260812 11:10:01.116940 29285 raft_consensus.cc:374] T 67e9a68c52554703ab56fada9459de86 P 9d36ca09efc643dd9d37dfedd9cd2403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:01.117002 29285 raft_consensus.cc:729] T 67e9a68c52554703ab56fada9459de86 P 9d36ca09efc643dd9d37dfedd9cd2403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d36ca09efc643dd9d37dfedd9cd2403, State: Initialized, Role: FOLLOWER
I20260812 11:10:01.117524 29285 consensus_queue.cc:260] T 67e9a68c52554703ab56fada9459de86 P 9d36ca09efc643dd9d37dfedd9cd2403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "084ef8e47f8e4b6d8eac7275ef273315" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 32917 } } peers { permanent_uuid: "ceb2d2ea9c474b06853d1891161296a8" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34889 } } peers { permanent_uuid: "9d36ca09efc643dd9d37dfedd9cd2403" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36081 } }
I20260812 11:10:01.118474 29285 ts_tablet_manager.cc:1428] T 67e9a68c52554703ab56fada9459de86 P 9d36ca09efc643dd9d37dfedd9cd2403: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 11:10:01.118846 28822 heartbeater.cc:503] Master 127.17.44.254:33165 was elected leader, sending a full tablet report...
I20260812 11:10:01.174093 29291 raft_consensus.cc:482] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:10:01.174365 29291 raft_consensus.cc:504] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "084ef8e47f8e4b6d8eac7275ef273315" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 32917 } } peers { permanent_uuid: "ceb2d2ea9c474b06853d1891161296a8" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34889 } } peers { permanent_uuid: "9d36ca09efc643dd9d37dfedd9cd2403" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36081 } }
I20260812 11:10:01.175845 29291 leader_election.cc:288] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 084ef8e47f8e4b6d8eac7275ef273315 (127.17.44.194:32917), 9d36ca09efc643dd9d37dfedd9cd2403 (127.17.44.193:36081)
I20260812 11:10:01.181636 29277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:01.176637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:34243 (local address 127.17.44.194:32917)
0812 11:10:01.176802 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:01.176807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:01.176823 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:10:01.176880 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:01.176883 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:01.176935 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:01.177033 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:01.177468 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.178290 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.180186 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.180793 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.180924 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:01.180998 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:01.181424 (+ 426us) server_negotiation.cc:299] Negotiation successful
0812 11:10:01.181506 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 11:10:01.181661 29276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:01.176852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:57433 (local address 127.17.44.193:36081)
0812 11:10:01.176987 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:01.176993 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:01.177010 (+ 17us) server_negotiation.cc:407] Connection header received
0812 11:10:01.177094 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:01.177097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:01.177149 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:01.177218 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:01.177525 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.178297 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.180186 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.180793 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.180923 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:01.180998 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:01.181421 (+ 423us) server_negotiation.cc:299] Negotiation successful
0812 11:10:01.181504 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 11:10:01.181905 29236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:01.176794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:36081 (local address 127.17.44.195:57433)
0812 11:10:01.176913 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:01.176928 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:01.177005 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:01.177296 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:01.177300 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:01.177324 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:01.177419 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:01.177427 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.178447 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.178451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:01.180011 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:01.180020 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.180960 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.180964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:01.181200 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:01.181255 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 11:10:01.181349 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 11:10:01.181460 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":190}
I20260812 11:10:01.181946 29235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:01.176398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:32917 (local address 127.17.44.195:34243)
0812 11:10:01.176553 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:01.176568 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:01.176666 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:01.177055 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:01.177061 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:01.177082 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:01.177336 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:01.177344 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.178447 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.178451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:01.180007 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:01.180020 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.180960 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.180964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:01.181195 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:01.181253 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 11:10:01.181349 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 11:10:01.181460 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":93}
I20260812 11:10:01.182802 28746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67e9a68c52554703ab56fada9459de86" candidate_uuid: "ceb2d2ea9c474b06853d1891161296a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d36ca09efc643dd9d37dfedd9cd2403" is_pre_election: true
I20260812 11:10:01.183238 28746 raft_consensus.cc:2454] T 67e9a68c52554703ab56fada9459de86 P 9d36ca09efc643dd9d37dfedd9cd2403 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb2d2ea9c474b06853d1891161296a8 in term 0.
I20260812 11:10:01.183393 28948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67e9a68c52554703ab56fada9459de86" candidate_uuid: "ceb2d2ea9c474b06853d1891161296a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "084ef8e47f8e4b6d8eac7275ef273315" is_pre_election: true
I20260812 11:10:01.183727 28948 raft_consensus.cc:2454] T 67e9a68c52554703ab56fada9459de86 P 084ef8e47f8e4b6d8eac7275ef273315 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb2d2ea9c474b06853d1891161296a8 in term 0.
I20260812 11:10:01.183764 29065 leader_election.cc:302] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d36ca09efc643dd9d37dfedd9cd2403, ceb2d2ea9c474b06853d1891161296a8; no voters:
I20260812 11:10:01.184116 29291 raft_consensus.cc:2781] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:10:01.184193 29291 raft_consensus.cc:482] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:10:01.184223 29291 raft_consensus.cc:3037] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:01.185575 29291 raft_consensus.cc:504] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "084ef8e47f8e4b6d8eac7275ef273315" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 32917 } } peers { permanent_uuid: "ceb2d2ea9c474b06853d1891161296a8" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34889 } } peers { permanent_uuid: "9d36ca09efc643dd9d37dfedd9cd2403" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36081 } }
I20260812 11:10:01.186117 29291 leader_election.cc:288] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8 [CANDIDATE]: Term 1 election: Requested vote from peers 084ef8e47f8e4b6d8eac7275ef273315 (127.17.44.194:32917), 9d36ca09efc643dd9d37dfedd9cd2403 (127.17.44.193:36081)
I20260812 11:10:01.186630 28948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67e9a68c52554703ab56fada9459de86" candidate_uuid: "ceb2d2ea9c474b06853d1891161296a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "084ef8e47f8e4b6d8eac7275ef273315"
I20260812 11:10:01.186754 28948 raft_consensus.cc:3037] T 67e9a68c52554703ab56fada9459de86 P 084ef8e47f8e4b6d8eac7275ef273315 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:01.186820 28746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67e9a68c52554703ab56fada9459de86" candidate_uuid: "ceb2d2ea9c474b06853d1891161296a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d36ca09efc643dd9d37dfedd9cd2403"
I20260812 11:10:01.187007 28746 raft_consensus.cc:3037] T 67e9a68c52554703ab56fada9459de86 P 9d36ca09efc643dd9d37dfedd9cd2403 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:01.188052 28948 raft_consensus.cc:2454] T 67e9a68c52554703ab56fada9459de86 P 084ef8e47f8e4b6d8eac7275ef273315 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb2d2ea9c474b06853d1891161296a8 in term 1.
I20260812 11:10:01.188479 29065 leader_election.cc:302] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 084ef8e47f8e4b6d8eac7275ef273315, ceb2d2ea9c474b06853d1891161296a8; no voters:
I20260812 11:10:01.188807 28746 raft_consensus.cc:2454] T 67e9a68c52554703ab56fada9459de86 P 9d36ca09efc643dd9d37dfedd9cd2403 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb2d2ea9c474b06853d1891161296a8 in term 1.
I20260812 11:10:01.188925 29291 raft_consensus.cc:2781] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:10:01.189213 29291 raft_consensus.cc:686] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8 [term 1 LEADER]: Becoming Leader. State: Replica: ceb2d2ea9c474b06853d1891161296a8, State: Running, Role: LEADER
I20260812 11:10:01.189579 29291 consensus_queue.cc:237] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "084ef8e47f8e4b6d8eac7275ef273315" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 32917 } } peers { permanent_uuid: "ceb2d2ea9c474b06853d1891161296a8" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34889 } } peers { permanent_uuid: "9d36ca09efc643dd9d37dfedd9cd2403" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36081 } }
I20260812 11:10:01.193521 27384 catalog_manager.cc:5495] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ceb2d2ea9c474b06853d1891161296a8 (127.17.44.195). New cstate: current_term: 1 leader_uuid: "ceb2d2ea9c474b06853d1891161296a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "084ef8e47f8e4b6d8eac7275ef273315" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 32917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb2d2ea9c474b06853d1891161296a8" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d36ca09efc643dd9d37dfedd9cd2403" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36081 } health_report { overall_health: UNKNOWN } } }
I20260812 11:10:01.209539 29280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:01.205299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49248 (local address 127.17.44.195:34889)
0812 11:10:01.205427 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:01.205431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:01.205446 (+ 15us) server_negotiation.cc:407] Connection header received
0812 11:10:01.205642 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:01.205644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:01.205705 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:01.205856 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:10:01.206197 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.206813 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.207864 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.208126 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.208240 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:01.208588 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:10:01.209060 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:10:01.209161 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:01.209292 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 11:10:01.209346 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 11:10:01.215618 28746 raft_consensus.cc:1261] T 67e9a68c52554703ab56fada9459de86 P 9d36ca09efc643dd9d37dfedd9cd2403 [term 1 FOLLOWER]: Refusing update from remote peer ceb2d2ea9c474b06853d1891161296a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:10:01.215719 28948 raft_consensus.cc:1261] T 67e9a68c52554703ab56fada9459de86 P 084ef8e47f8e4b6d8eac7275ef273315 [term 1 FOLLOWER]: Refusing update from remote peer ceb2d2ea9c474b06853d1891161296a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:10:01.217051 29295 consensus_queue.cc:1035] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d36ca09efc643dd9d37dfedd9cd2403" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:10:01.217402 29291 consensus_queue.cc:1035] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "084ef8e47f8e4b6d8eac7275ef273315" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 32917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:10:01.227058 29299 mvcc.cc:205] Tried to move back new op lower bound from 7317639172971339776 to 7317639172875001856. Current Snapshot: MvccSnapshot[applied={T|T < 7317639172971339776}]
I20260812 11:10:01.228899 29300 mvcc.cc:205] Tried to move back new op lower bound from 7317639172971339776 to 7317639172875001856. Current Snapshot: MvccSnapshot[applied={T|T < 7317639172971339776}]
I20260812 11:10:01.230245 29301 mvcc.cc:205] Tried to move back new op lower bound from 7317639172971339776 to 7317639172875001856. Current Snapshot: MvccSnapshot[applied={T|T < 7317639172971339776}]
I20260812 11:10:01.241065 27384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:10:01.241264 27384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:10:01.246718 27384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fac979aa6d9a4662a500f789ee828ae3: Sending DeleteTablet for 3 replicas of tablet 67e9a68c52554703ab56fada9459de86
I20260812 11:10:01.247785 28716 tablet_service.cc:1514] Processing DeleteTablet for tablet 67e9a68c52554703ab56fada9459de86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:10:01 UTC) from {username='oryx', principal='oryx/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:42798
I20260812 11:10:01.248476 29309 tablet_replica.cc:330] T 67e9a68c52554703ab56fada9459de86 P 9d36ca09efc643dd9d37dfedd9cd2403: stopping tablet replica
I20260812 11:10:01.248605 28923 tablet_service.cc:1514] Processing DeleteTablet for tablet 67e9a68c52554703ab56fada9459de86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:10:01 UTC) from {username='oryx', principal='oryx/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:54196
I20260812 11:10:01.248824 29309 raft_consensus.cc:2229] T 67e9a68c52554703ab56fada9459de86 P 9d36ca09efc643dd9d37dfedd9cd2403 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:10:01.248793 29132 tablet_service.cc:1514] Processing DeleteTablet for tablet 67e9a68c52554703ab56fada9459de86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:10:01 UTC) from {username='oryx', principal='oryx/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:49244
I20260812 11:10:01.249162 29309 raft_consensus.cc:2258] T 67e9a68c52554703ab56fada9459de86 P 9d36ca09efc643dd9d37dfedd9cd2403 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:10:01.249429 29313 tablet_replica.cc:330] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8: stopping tablet replica
I20260812 11:10:01.249651 29311 tablet_replica.cc:330] T 67e9a68c52554703ab56fada9459de86 P 084ef8e47f8e4b6d8eac7275ef273315: stopping tablet replica
I20260812 11:10:01.250010 29311 raft_consensus.cc:2229] T 67e9a68c52554703ab56fada9459de86 P 084ef8e47f8e4b6d8eac7275ef273315 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:10:01.250337 29313 raft_consensus.cc:2229] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:10:01.250768 29313 raft_consensus.cc:2258] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:10:01.251042 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 27339
I20260812 11:10:01.258299 29309 ts_tablet_manager.cc:1905] T 67e9a68c52554703ab56fada9459de86 P 9d36ca09efc643dd9d37dfedd9cd2403: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:10:01.259568 29313 ts_tablet_manager.cc:1905] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:10:01.263041 29309 ts_tablet_manager.cc:1918] T 67e9a68c52554703ab56fada9459de86 P 9d36ca09efc643dd9d37dfedd9cd2403: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:10:01.263166 29309 log.cc:1196] T 67e9a68c52554703ab56fada9459de86 P 9d36ca09efc643dd9d37dfedd9cd2403: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-0/wal/wals/67e9a68c52554703ab56fada9459de86
I20260812 11:10:01.263597 29309 ts_tablet_manager.cc:1939] T 67e9a68c52554703ab56fada9459de86 P 9d36ca09efc643dd9d37dfedd9cd2403: Deleting consensus metadata
I20260812 11:10:01.264019 29313 ts_tablet_manager.cc:1918] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:10:01.264202 29313 log.cc:1196] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/67e9a68c52554703ab56fada9459de86
I20260812 11:10:01.264606 29313 ts_tablet_manager.cc:1939] T 67e9a68c52554703ab56fada9459de86 P ceb2d2ea9c474b06853d1891161296a8: Deleting consensus metadata
I20260812 11:10:01.265576 29311 raft_consensus.cc:2258] T 67e9a68c52554703ab56fada9459de86 P 084ef8e47f8e4b6d8eac7275ef273315 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:10:01.267409 29311 ts_tablet_manager.cc:1905] T 67e9a68c52554703ab56fada9459de86 P 084ef8e47f8e4b6d8eac7275ef273315: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 11:10:01.270315 29067 connection.cc:535] server connection from 127.0.0.1:49244 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 11:10:01.270506 28648 connection.cc:535] server connection from 127.0.0.1:42798 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 11:10:01.271461 29311 ts_tablet_manager.cc:1918] T 67e9a68c52554703ab56fada9459de86 P 084ef8e47f8e4b6d8eac7275ef273315: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:10:01.271586 29311 log.cc:1196] T 67e9a68c52554703ab56fada9459de86 P 084ef8e47f8e4b6d8eac7275ef273315: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/67e9a68c52554703ab56fada9459de86
I20260812 11:10:01.271916 29311 ts_tablet_manager.cc:1939] T 67e9a68c52554703ab56fada9459de86 P 084ef8e47f8e4b6d8eac7275ef273315: Deleting consensus metadata
W20260812 11:10:01.273033 28857 connection.cc:441] server connection from 127.0.0.1:54196 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54196 (request call id 1) could send its response
I20260812 11:10:01.281092 28457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:01.251150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45064 (local address 127.17.44.253:35581)
0812 11:10:01.253589 (+ 2439us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:01.253594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:01.261261 (+ 7667us) server_negotiation.cc:407] Connection header received
0812 11:10:01.262172 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:01.262176 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:01.262236 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:01.262289 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:01.266131 (+ 3842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.266918 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.268867 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.269230 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.269337 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:01.275924 (+ 6587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:01.275928 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:01.275931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:01.275969 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:01.277807 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:01.278324 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:01.278328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:01.278330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:01.278424 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:01.278718 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:01.278722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:01.278727 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:01.278970 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:01.279104 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:01.280212 (+ 1108us) server_negotiation.cc:299] Negotiation successful
0812 11:10:01.280603 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2348}
I20260812 11:10:01.285647 29249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:01.275039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34162 (local address 127.17.44.252:33565)
0812 11:10:01.275179 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:01.275184 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:01.275944 (+ 760us) server_negotiation.cc:407] Connection header received
0812 11:10:01.276163 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:01.276167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:01.276220 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:01.276276 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:01.277247 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.278187 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.279449 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.279817 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.279893 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:01.282045 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:01.282049 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:01.282052 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:01.282077 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:01.283824 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:01.284329 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:01.284333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:01.284335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:01.284430 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:01.284724 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:01.284727 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:01.284729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:01.284928 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:01.285028 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:01.285359 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 11:10:01.285491 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 11:10:01.288139 29243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:01.274915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45074 (local address 127.17.44.253:35581)
0812 11:10:01.275102 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:01.275109 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:01.276016 (+ 907us) server_negotiation.cc:407] Connection header received
0812 11:10:01.276593 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:01.276598 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:01.276653 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:01.276711 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:01.277750 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.278577 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.279783 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.280144 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.280215 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:01.283229 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:01.283233 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:01.283235 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:01.283260 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:01.285338 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:01.286231 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:01.286236 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:01.286239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:01.286339 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:01.286912 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:01.286918 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:01.286923 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:01.287306 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:01.287456 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:01.287851 (+ 395us) server_negotiation.cc:299] Negotiation successful
0812 11:10:01.287988 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260812 11:10:01.289988 28833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:01.251157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34152 (local address 127.17.44.252:33565)
0812 11:10:01.256654 (+ 5497us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:01.256661 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:01.279926 (+ 23265us) server_negotiation.cc:407] Connection header received
0812 11:10:01.279965 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:01.279968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:01.280011 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:01.280078 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:01.281514 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.282209 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.283558 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:01.283956 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:01.284071 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:01.285905 (+ 1834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:01.285909 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:01.285912 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:01.285936 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:01.287961 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:01.288490 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:01.288493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:01.288495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:01.288586 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:01.289225 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:01.289228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:01.289230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:01.289396 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:01.289496 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:01.289752 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 11:10:01.289842 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5368}
I20260812 11:10:02.236949 29326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:02.236431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.194:47853)
0812 11:10:02.236712 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:02.236817 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":139,"threads_started":1}
I20260812 11:10:02.237030 29327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:02.236473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.193:40139)
0812 11:10:02.236847 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:02.236930 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":127,"threads_started":1}
W20260812 11:10:02.237287 29025 heartbeater.cc:644] Failed to heartbeat to 127.17.44.254:33165 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
W20260812 11:10:02.237529 28822 heartbeater.cc:644] Failed to heartbeat to 127.17.44.254:33165 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
I20260812 11:10:02.237706 29328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:02.237265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.195:54625)
0812 11:10:02.237496 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:02.237606 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":75,"threads_started":1}
W20260812 11:10:02.237963 29230 heartbeater.cc:644] Failed to heartbeat to 127.17.44.254:33165 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
I20260812 11:10:02.746788 29330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fac979aa6d9a4662a500f789ee828ae3)
I20260812 11:10:02.746815 29329 raft_consensus.cc:482] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fac979aa6d9a4662a500f789ee828ae3)
I20260812 11:10:02.746996 29330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: false } }
I20260812 11:10:02.747117 29329 raft_consensus.cc:504] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: false } }
I20260812 11:10:02.747879 29330 leader_election.cc:288] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fac979aa6d9a4662a500f789ee828ae3 (127.17.44.254:33165), 395bb06cf1e84d9ba73408995fc72721 (127.17.44.253:35581)
I20260812 11:10:02.748365 29329 leader_election.cc:288] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fac979aa6d9a4662a500f789ee828ae3 (127.17.44.254:33165), 57d679a7e23e4081b1f929452563ca04 (127.17.44.252:33565)
I20260812 11:10:02.748420 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:02.747851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.0.0.1:35224)
0812 11:10:02.748205 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:02.748300 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":75,"threads_started":1}
W20260812 11:10:02.748596 28527 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
I20260812 11:10:02.749286 29334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:02.748419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.0.0.1:35240)
0812 11:10:02.749097 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:02.749173 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":91,"threads_started":1}
W20260812 11:10:02.749482 28259 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
I20260812 11:10:02.749573 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:02.749289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.0.0.1:35252)
0812 11:10:02.749436 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:02.749482 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260812 11:10:02.749733 28527 leader_election.cc:334] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fac979aa6d9a4662a500f789ee828ae3 (127.17.44.254:33165): Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
I20260812 11:10:02.750465 29334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:02.750254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.0.0.1:35266)
0812 11:10:02.750340 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:02.750395 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 11:10:02.750628 28259 leader_election.cc:334] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fac979aa6d9a4662a500f789ee828ae3 (127.17.44.254:33165): Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
I20260812 11:10:02.753043 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:02.747851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:35581 (local address 127.0.0.1:45088)
0812 11:10:02.748240 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:02.748253 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:02.748882 (+ 629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:02.749222 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:02.749228 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:02.749251 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:02.749375 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:02.749382 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:02.750567 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:02.750570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:02.751677 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:02.751687 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:02.752631 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:02.752634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:02.752723 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:02.752751 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 11:10:02.752845 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 11:10:02.752914 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":169,"thread_start_us":47,"threads_started":1}
I20260812 11:10:02.753456 29333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:02.747978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45088 (local address 127.17.44.253:35581)
0812 11:10:02.748296 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:02.748301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:02.748827 (+ 526us) server_negotiation.cc:407] Connection header received
0812 11:10:02.749007 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:02.749014 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:02.749079 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:02.749497 (+ 418us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:02.749618 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:02.750406 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:02.751814 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:02.752457 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:02.752586 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:02.752654 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:02.753008 (+ 354us) server_negotiation.cc:299] Negotiation successful
0812 11:10:02.753241 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":88,"threads_started":1}
I20260812 11:10:02.753831 29335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:02.748566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:33565 (local address 127.0.0.1:34170)
0812 11:10:02.749331 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:02.749436 (+ 105us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:02.749574 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:02.750076 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:02.750079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:02.750096 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:02.750171 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:02.750176 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:02.751286 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:02.751289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:02.752472 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:02.752481 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:02.753396 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:02.753400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:02.753508 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:02.753530 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 11:10:02.753620 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 11:10:02.753699 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"spinlock_wait_cycles":896,"thread_start_us":61,"threads_started":1}
I20260812 11:10:02.753841 29337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:02.748896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34170 (local address 127.17.44.252:33565)
0812 11:10:02.749203 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:02.749212 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:02.749498 (+ 286us) server_negotiation.cc:407] Connection header received
0812 11:10:02.749698 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:02.749701 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:02.749741 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:02.749794 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:02.750292 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:02.751140 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:02.752617 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:02.753250 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:02.753381 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:02.753434 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:02.753650 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 11:10:02.753719 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":124,"threads_started":1}
I20260812 11:10:02.754245 28293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "57d679a7e23e4081b1f929452563ca04" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "395bb06cf1e84d9ba73408995fc72721" is_pre_election: true
I20260812 11:10:02.754470 28293 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57d679a7e23e4081b1f929452563ca04 in term 1.
I20260812 11:10:02.754454 28554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "395bb06cf1e84d9ba73408995fc72721" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "57d679a7e23e4081b1f929452563ca04" is_pre_election: true
I20260812 11:10:02.754730 28554 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 395bb06cf1e84d9ba73408995fc72721 in term 1.
I20260812 11:10:02.754876 28524 leader_election.cc:302] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 395bb06cf1e84d9ba73408995fc72721, 57d679a7e23e4081b1f929452563ca04; no voters: fac979aa6d9a4662a500f789ee828ae3
I20260812 11:10:02.755131 29330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 11:10:02.755218 29330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fac979aa6d9a4662a500f789ee828ae3)
I20260812 11:10:02.755275 29330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 1 FOLLOWER]: Advancing to term 2
I20260812 11:10:02.756803 29330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: false } }
I20260812 11:10:02.757213 29330 leader_election.cc:288] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [CANDIDATE]: Term 2 election: Requested vote from peers fac979aa6d9a4662a500f789ee828ae3 (127.17.44.254:33165), 395bb06cf1e84d9ba73408995fc72721 (127.17.44.253:35581)
I20260812 11:10:02.757246 28258 leader_election.cc:302] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 395bb06cf1e84d9ba73408995fc72721, 57d679a7e23e4081b1f929452563ca04; no voters: fac979aa6d9a4662a500f789ee828ae3
I20260812 11:10:02.757546 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:02.757336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.0.0.1:35268)
0812 11:10:02.757431 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:02.757482 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 11:10:02.757532 29329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 11:10:02.757588 29329 raft_consensus.cc:482] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fac979aa6d9a4662a500f789ee828ae3)
I20260812 11:10:02.757586 28293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "57d679a7e23e4081b1f929452563ca04" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "395bb06cf1e84d9ba73408995fc72721"
I20260812 11:10:02.757676 29329 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 1 FOLLOWER]: Advancing to term 2
I20260812 11:10:02.758464 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:02.758150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.0.0.1:35280)
0812 11:10:02.758335 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:02.758384 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":81}
W20260812 11:10:02.758603 28527 leader_election.cc:334] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer fac979aa6d9a4662a500f789ee828ae3 (127.17.44.254:33165): Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
I20260812 11:10:02.759130 29329 raft_consensus.cc:504] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: false } }
I20260812 11:10:02.759519 29329 leader_election.cc:288] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [CANDIDATE]: Term 2 election: Requested vote from peers fac979aa6d9a4662a500f789ee828ae3 (127.17.44.254:33165), 57d679a7e23e4081b1f929452563ca04 (127.17.44.252:33565)
I20260812 11:10:02.759943 28554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "395bb06cf1e84d9ba73408995fc72721" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "57d679a7e23e4081b1f929452563ca04"
I20260812 11:10:02.760133 28554 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 395bb06cf1e84d9ba73408995fc72721 in current term 2: Already voted for candidate 57d679a7e23e4081b1f929452563ca04 in this term.
I20260812 11:10:02.760181 28293 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 57d679a7e23e4081b1f929452563ca04 in current term 2: Already voted for candidate 395bb06cf1e84d9ba73408995fc72721 in this term.
I20260812 11:10:02.760303 29335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:02.760014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.0.0.1:35296)
0812 11:10:02.760151 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:02.760205 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 11:10:02.760776 28524 leader_election.cc:302] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57d679a7e23e4081b1f929452563ca04; no voters: 395bb06cf1e84d9ba73408995fc72721, fac979aa6d9a4662a500f789ee828ae3
I20260812 11:10:02.760972 29330 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 11:10:02.761591 29335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:02.761228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.0.0.1:35304)
0812 11:10:02.761454 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:02.761503 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
W20260812 11:10:02.761735 28259 leader_election.cc:334] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer fac979aa6d9a4662a500f789ee828ae3 (127.17.44.254:33165): Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
I20260812 11:10:02.761832 28259 leader_election.cc:302] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 395bb06cf1e84d9ba73408995fc72721; no voters: 57d679a7e23e4081b1f929452563ca04, fac979aa6d9a4662a500f789ee828ae3
I20260812 11:10:02.761992 29329 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 11:10:03.238333 29340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:03.237868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.194:43715)
0812 11:10:03.238138 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:03.238206 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":71,"threads_started":1}
I20260812 11:10:03.238759 29342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:03.238046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.193:36943)
0812 11:10:03.238482 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:03.238591 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":102,"threads_started":1}
I20260812 11:10:03.239454 29343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:03.238808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.195:59017)
0812 11:10:03.239315 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:03.239374 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":243,"threads_started":1}
I20260812 11:10:04.239652 29344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:04.239214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.194:38583)
0812 11:10:04.239512 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:04.239565 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":69,"threads_started":1}
I20260812 11:10:04.240231 29345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:04.239661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.193:58247)
0812 11:10:04.240092 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:04.240146 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":89,"threads_started":1}
I20260812 11:10:04.240483 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:04.240090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.195:33143)
0812 11:10:04.240313 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:04.240390 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":79,"threads_started":1}
I20260812 11:10:04.704433 29348 raft_consensus.cc:482] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:10:04.704617 29348 raft_consensus.cc:504] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: false } }
I20260812 11:10:04.705081 29348 leader_election.cc:288] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fac979aa6d9a4662a500f789ee828ae3 (127.17.44.254:33165), 57d679a7e23e4081b1f929452563ca04 (127.17.44.252:33565)
I20260812 11:10:04.705844 28554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "395bb06cf1e84d9ba73408995fc72721" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "57d679a7e23e4081b1f929452563ca04" is_pre_election: true
I20260812 11:10:04.706019 28554 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 395bb06cf1e84d9ba73408995fc72721 in term 2.
I20260812 11:10:04.706193 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:04.705635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.0.0.1:50084)
0812 11:10:04.706005 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:04.706088 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":93,"threads_started":1}
I20260812 11:10:04.706439 28258 leader_election.cc:302] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 395bb06cf1e84d9ba73408995fc72721, 57d679a7e23e4081b1f929452563ca04; no voters:
I20260812 11:10:04.706619 29348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 11:10:04.706691 29348 raft_consensus.cc:482] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:10:04.706734 29348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 2 FOLLOWER]: Advancing to term 3
I20260812 11:10:04.707154 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:04.706868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.0.0.1:50086)
0812 11:10:04.707029 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:04.707083 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 11:10:04.707322 28259 leader_election.cc:334] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fac979aa6d9a4662a500f789ee828ae3 (127.17.44.254:33165): Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
I20260812 11:10:04.708067 29348 raft_consensus.cc:504] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: false } }
I20260812 11:10:04.708437 29348 leader_election.cc:288] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [CANDIDATE]: Term 3 election: Requested vote from peers fac979aa6d9a4662a500f789ee828ae3 (127.17.44.254:33165), 57d679a7e23e4081b1f929452563ca04 (127.17.44.252:33565)
I20260812 11:10:04.708807 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:04.708578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.0.0.1:50098)
0812 11:10:04.708705 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:04.708759 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 11:10:04.708791 28554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "395bb06cf1e84d9ba73408995fc72721" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "57d679a7e23e4081b1f929452563ca04"
I20260812 11:10:04.708920 28554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 2 FOLLOWER]: Advancing to term 3
I20260812 11:10:04.709475 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:04.709317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.0.0.1:50106)
0812 11:10:04.709384 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:04.709424 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260812 11:10:04.709637 28259 leader_election.cc:334] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer fac979aa6d9a4662a500f789ee828ae3 (127.17.44.254:33165): Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
I20260812 11:10:04.710201 28554 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 395bb06cf1e84d9ba73408995fc72721 in term 3.
I20260812 11:10:04.710493 28258 leader_election.cc:302] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 395bb06cf1e84d9ba73408995fc72721, 57d679a7e23e4081b1f929452563ca04; no voters: fac979aa6d9a4662a500f789ee828ae3
I20260812 11:10:04.710664 29348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 11:10:04.710845 29348 raft_consensus.cc:686] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [term 3 LEADER]: Becoming Leader. State: Replica: 395bb06cf1e84d9ba73408995fc72721, State: Running, Role: LEADER
I20260812 11:10:04.711158 29348 consensus_queue.cc:237] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: false } }
I20260812 11:10:04.712285 29352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 395bb06cf1e84d9ba73408995fc72721. Latest consensus state: current_term: 3 leader_uuid: "395bb06cf1e84d9ba73408995fc72721" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: false } } }
I20260812 11:10:04.712431 29352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [sys.catalog]: This master's current role is: LEADER
I20260812 11:10:04.712729 29354 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:10:04.715778 29354 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:10:04.716244 29354 catalog_manager.cc:1259] Loaded cluster ID: 7332444ac4b746989b256ca4915388fd
I20260812 11:10:04.716315 29354 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:10:04.716938 29354 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:10:04.717362 29354 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721: Loaded TSK: 0
I20260812 11:10:04.718052 29354 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:10:04.774194 28277 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45074:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:10:04.777657 28277 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:10:04.781244 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:04.780941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.0.0.1:50116)
0812 11:10:04.781121 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:04.781176 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 11:10:04.781330 28554 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [term 3 FOLLOWER]: Refusing update from remote peer 395bb06cf1e84d9ba73408995fc72721: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260812 11:10:04.781878 29352 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260812 11:10:04.782088 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:04.781851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.0.0.1:50126)
0812 11:10:04.781957 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:04.782001 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 11:10:04.782212 28259 consensus_peers.cc:487] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 -> Peer fac979aa6d9a4662a500f789ee828ae3 (127.17.44.254:33165): Couldn't send request to peer fac979aa6d9a4662a500f789ee828ae3. Status: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 11:10:04.783927 29356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 395bb06cf1e84d9ba73408995fc72721. Latest consensus state: current_term: 3 leader_uuid: "395bb06cf1e84d9ba73408995fc72721" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: false } } }
I20260812 11:10:04.784179 29356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:10:04.785785 29348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "395bb06cf1e84d9ba73408995fc72721" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: false } } }
I20260812 11:10:04.785916 29348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721 [sys.catalog]: This master's current role is: LEADER
I20260812 11:10:04.786305 29356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "395bb06cf1e84d9ba73408995fc72721" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fac979aa6d9a4662a500f789ee828ae3" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33165 } } peers { permanent_uuid: "395bb06cf1e84d9ba73408995fc72721" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "57d679a7e23e4081b1f929452563ca04" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 33565 } attrs { promote: false } } }
I20260812 11:10:04.786446 29356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57d679a7e23e4081b1f929452563ca04 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:10:04.786432 29357 mvcc.cc:205] Tried to move back new op lower bound from 7317639187576995840 to 7317639187299471360. Current Snapshot: MvccSnapshot[applied={T|T < 7317639187576995840}]
I20260812 11:10:04.798681 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:04.793570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:32917 (local address 127.0.0.1:38714)
0812 11:10:04.794080 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:04.794103 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:04.794235 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:04.794598 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:04.794601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:04.794625 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:04.794744 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:04.794752 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:04.795905 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:04.795909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:04.797255 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:04.797266 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:04.798154 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:04.798158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:04.798277 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:04.798309 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 11:10:04.798396 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 11:10:04.798503 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":31}
I20260812 11:10:04.798805 29365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:04.794096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35640 (local address 127.17.44.195:34889)
0812 11:10:04.794421 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:04.794425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:04.794856 (+ 431us) server_negotiation.cc:407] Connection header received
0812 11:10:04.794950 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:04.794953 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:04.795013 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:04.795104 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:04.795394 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:04.796109 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:04.797627 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:04.798192 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:04.798300 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:04.798369 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:04.798583 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 11:10:04.798660 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":109,"threads_started":1}
I20260812 11:10:04.798851 29363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:04.793860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:34889 (local address 127.0.0.1:35640)
0812 11:10:04.794638 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:04.794655 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:04.794759 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:04.795142 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:04.795146 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:04.795164 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:04.795275 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:04.795284 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:04.796232 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:04.796235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:04.797490 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:04.797498 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:04.798322 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:04.798325 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:04.798413 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:04.798446 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 11:10:04.798552 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 11:10:04.798644 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"mutex_wait_us":24,"thread_start_us":115,"threads_started":1}
I20260812 11:10:04.799055 29366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:04.794197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:36081 (local address 127.0.0.1:42590)
0812 11:10:04.794738 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:04.794750 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:04.794821 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:04.795348 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:04.795354 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:04.795373 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:04.795450 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:04.795456 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:04.796828 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:04.796833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:04.797651 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:04.797662 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:04.798670 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:04.798674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:04.798762 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:04.798784 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 11:10:04.798835 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 11:10:04.798907 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":114,"threads_started":1}
I20260812 11:10:04.799544 29367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:04.794647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42590 (local address 127.17.44.193:36081)
0812 11:10:04.795094 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:04.795100 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:04.795117 (+ 17us) server_negotiation.cc:407] Connection header received
0812 11:10:04.795169 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:04.795172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:04.795216 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:04.795558 (+ 342us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:04.795680 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:04.796659 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:04.797819 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:04.798539 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:04.799247 (+ 708us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:04.799314 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:04.799350 (+ 36us) server_negotiation.cc:299] Negotiation successful
0812 11:10:04.799418 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":71,"threads_started":1}
I20260812 11:10:04.799485 29132 tablet_service.cc:1467] Processing CreateTablet for tablet 0ddcf87fae354827a8ed1d5c9b052e88 (DEFAULT_TABLE table=test-table [id=f9f89fe8061c46efbc5c0a8b83ec85ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:10:04.799830 29132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ddcf87fae354827a8ed1d5c9b052e88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:04.800028 28716 tablet_service.cc:1467] Processing CreateTablet for tablet 0ddcf87fae354827a8ed1d5c9b052e88 (DEFAULT_TABLE table=test-table [id=f9f89fe8061c46efbc5c0a8b83ec85ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:10:04.800323 28716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ddcf87fae354827a8ed1d5c9b052e88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:04.800411 29362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:04.793769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38714 (local address 127.17.44.194:32917)
0812 11:10:04.794218 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:04.794224 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:04.794243 (+ 19us) server_negotiation.cc:407] Connection header received
0812 11:10:04.794350 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:04.794355 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:04.794403 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:04.794521 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:04.794913 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:04.795758 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:04.797405 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:04.798006 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:04.798122 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:04.798185 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:04.799922 (+ 1737us) server_negotiation.cc:299] Negotiation successful
0812 11:10:04.800027 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":117,"threads_started":1}
I20260812 11:10:04.800928 28923 tablet_service.cc:1467] Processing CreateTablet for tablet 0ddcf87fae354827a8ed1d5c9b052e88 (DEFAULT_TABLE table=test-table [id=f9f89fe8061c46efbc5c0a8b83ec85ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:10:04.801184 28923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ddcf87fae354827a8ed1d5c9b052e88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:04.802161 29369 tablet_bootstrap.cc:492] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8: Bootstrap starting.
I20260812 11:10:04.803205 29369 tablet_bootstrap.cc:654] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:04.803253 29370 tablet_bootstrap.cc:492] T 0ddcf87fae354827a8ed1d5c9b052e88 P 9d36ca09efc643dd9d37dfedd9cd2403: Bootstrap starting.
I20260812 11:10:04.804118 29373 tablet_bootstrap.cc:492] T 0ddcf87fae354827a8ed1d5c9b052e88 P 084ef8e47f8e4b6d8eac7275ef273315: Bootstrap starting.
I20260812 11:10:04.804278 29370 tablet_bootstrap.cc:654] T 0ddcf87fae354827a8ed1d5c9b052e88 P 9d36ca09efc643dd9d37dfedd9cd2403: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:04.804682 29369 tablet_bootstrap.cc:492] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8: No bootstrap required, opened a new log
I20260812 11:10:04.804852 29369 ts_tablet_manager.cc:1397] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 11:10:04.805068 29373 tablet_bootstrap.cc:654] T 0ddcf87fae354827a8ed1d5c9b052e88 P 084ef8e47f8e4b6d8eac7275ef273315: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:04.805334 29369 raft_consensus.cc:348] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "084ef8e47f8e4b6d8eac7275ef273315" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 32917 } } peers { permanent_uuid: "ceb2d2ea9c474b06853d1891161296a8" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34889 } } peers { permanent_uuid: "9d36ca09efc643dd9d37dfedd9cd2403" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36081 } }
I20260812 11:10:04.805465 29369 raft_consensus.cc:374] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:04.805502 29369 raft_consensus.cc:729] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb2d2ea9c474b06853d1891161296a8, State: Initialized, Role: FOLLOWER
I20260812 11:10:04.805601 29370 tablet_bootstrap.cc:492] T 0ddcf87fae354827a8ed1d5c9b052e88 P 9d36ca09efc643dd9d37dfedd9cd2403: No bootstrap required, opened a new log
I20260812 11:10:04.805665 29369 consensus_queue.cc:260] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "084ef8e47f8e4b6d8eac7275ef273315" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 32917 } } peers { permanent_uuid: "ceb2d2ea9c474b06853d1891161296a8" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34889 } } peers { permanent_uuid: "9d36ca09efc643dd9d37dfedd9cd2403" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36081 } }
I20260812 11:10:04.805742 29370 ts_tablet_manager.cc:1397] T 0ddcf87fae354827a8ed1d5c9b052e88 P 9d36ca09efc643dd9d37dfedd9cd2403: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 11:10:04.806051 29369 ts_tablet_manager.cc:1428] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:10:04.806228 29370 raft_consensus.cc:348] T 0ddcf87fae354827a8ed1d5c9b052e88 P 9d36ca09efc643dd9d37dfedd9cd2403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "084ef8e47f8e4b6d8eac7275ef273315" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 32917 } } peers { permanent_uuid: "ceb2d2ea9c474b06853d1891161296a8" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34889 } } peers { permanent_uuid: "9d36ca09efc643dd9d37dfedd9cd2403" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36081 } }
I20260812 11:10:04.806334 29370 raft_consensus.cc:374] T 0ddcf87fae354827a8ed1d5c9b052e88 P 9d36ca09efc643dd9d37dfedd9cd2403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:04.806380 29370 raft_consensus.cc:729] T 0ddcf87fae354827a8ed1d5c9b052e88 P 9d36ca09efc643dd9d37dfedd9cd2403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d36ca09efc643dd9d37dfedd9cd2403, State: Initialized, Role: FOLLOWER
I20260812 11:10:04.806381 29373 tablet_bootstrap.cc:492] T 0ddcf87fae354827a8ed1d5c9b052e88 P 084ef8e47f8e4b6d8eac7275ef273315: No bootstrap required, opened a new log
I20260812 11:10:04.806463 29373 ts_tablet_manager.cc:1397] T 0ddcf87fae354827a8ed1d5c9b052e88 P 084ef8e47f8e4b6d8eac7275ef273315: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 11:10:04.806542 29370 consensus_queue.cc:260] T 0ddcf87fae354827a8ed1d5c9b052e88 P 9d36ca09efc643dd9d37dfedd9cd2403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "084ef8e47f8e4b6d8eac7275ef273315" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 32917 } } peers { permanent_uuid: "ceb2d2ea9c474b06853d1891161296a8" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34889 } } peers { permanent_uuid: "9d36ca09efc643dd9d37dfedd9cd2403" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36081 } }
I20260812 11:10:04.806907 29370 ts_tablet_manager.cc:1428] T 0ddcf87fae354827a8ed1d5c9b052e88 P 9d36ca09efc643dd9d37dfedd9cd2403: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:10:04.806871 29373 raft_consensus.cc:348] T 0ddcf87fae354827a8ed1d5c9b052e88 P 084ef8e47f8e4b6d8eac7275ef273315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "084ef8e47f8e4b6d8eac7275ef273315" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 32917 } } peers { permanent_uuid: "ceb2d2ea9c474b06853d1891161296a8" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34889 } } peers { permanent_uuid: "9d36ca09efc643dd9d37dfedd9cd2403" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36081 } }
I20260812 11:10:04.807011 29373 raft_consensus.cc:374] T 0ddcf87fae354827a8ed1d5c9b052e88 P 084ef8e47f8e4b6d8eac7275ef273315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:04.807057 29373 raft_consensus.cc:729] T 0ddcf87fae354827a8ed1d5c9b052e88 P 084ef8e47f8e4b6d8eac7275ef273315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 084ef8e47f8e4b6d8eac7275ef273315, State: Initialized, Role: FOLLOWER
I20260812 11:10:04.807179 29373 consensus_queue.cc:260] T 0ddcf87fae354827a8ed1d5c9b052e88 P 084ef8e47f8e4b6d8eac7275ef273315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "084ef8e47f8e4b6d8eac7275ef273315" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 32917 } } peers { permanent_uuid: "ceb2d2ea9c474b06853d1891161296a8" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34889 } } peers { permanent_uuid: "9d36ca09efc643dd9d37dfedd9cd2403" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36081 } }
I20260812 11:10:04.807518 29373 ts_tablet_manager.cc:1428] T 0ddcf87fae354827a8ed1d5c9b052e88 P 084ef8e47f8e4b6d8eac7275ef273315: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:10:04.810555 29386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:04.808367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.193:48367)
0812 11:10:04.810195 (+ 1828us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:04.810258 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1743,"thread_start_us":99,"threads_started":1}
I20260812 11:10:04.810544 29231 heartbeater.cc:503] Master 127.17.44.253:35581 was elected leader, sending a full tablet report...
I20260812 11:10:04.811414 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:04.806438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.195:55123)
0812 11:10:04.810792 (+ 4354us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:04.810853 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4287,"thread_start_us":101,"threads_started":1}
I20260812 11:10:04.811668 29385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:04.808300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.194:38081)
0812 11:10:04.810595 (+ 2295us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:04.810679 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2226,"thread_start_us":101,"threads_started":1}
I20260812 11:10:04.812435 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:04.812154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.195:56661)
0812 11:10:04.812318 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:04.812371 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260812 11:10:04.812489 29385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:04.812227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.194:34047)
0812 11:10:04.812369 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:04.812423 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 11:10:04.812564 28824 heartbeater.cc:503] Master 127.17.44.253:35581 was elected leader, sending a full tablet report...
I20260812 11:10:04.812821 29029 heartbeater.cc:503] Master 127.17.44.253:35581 was elected leader, sending a full tablet report...
I20260812 11:10:04.814664 29386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:04.814327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.193:45601)
0812 11:10:04.814538 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:04.814595 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
W20260812 11:10:05.018947 29232 tablet.cc:2369] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:10:05.028604 29379 raft_consensus.cc:482] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:10:05.028782 29379 raft_consensus.cc:504] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "084ef8e47f8e4b6d8eac7275ef273315" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 32917 } } peers { permanent_uuid: "ceb2d2ea9c474b06853d1891161296a8" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34889 } } peers { permanent_uuid: "9d36ca09efc643dd9d37dfedd9cd2403" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36081 } }
I20260812 11:10:05.029454 29379 leader_election.cc:288] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 084ef8e47f8e4b6d8eac7275ef273315 (127.17.44.194:32917), 9d36ca09efc643dd9d37dfedd9cd2403 (127.17.44.193:36081)
I20260812 11:10:05.030217 28948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ddcf87fae354827a8ed1d5c9b052e88" candidate_uuid: "ceb2d2ea9c474b06853d1891161296a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "084ef8e47f8e4b6d8eac7275ef273315" is_pre_election: true
I20260812 11:10:05.030412 28948 raft_consensus.cc:2454] T 0ddcf87fae354827a8ed1d5c9b052e88 P 084ef8e47f8e4b6d8eac7275ef273315 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb2d2ea9c474b06853d1891161296a8 in term 0.
I20260812 11:10:05.030777 28746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ddcf87fae354827a8ed1d5c9b052e88" candidate_uuid: "ceb2d2ea9c474b06853d1891161296a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d36ca09efc643dd9d37dfedd9cd2403" is_pre_election: true
I20260812 11:10:05.030989 28746 raft_consensus.cc:2454] T 0ddcf87fae354827a8ed1d5c9b052e88 P 9d36ca09efc643dd9d37dfedd9cd2403 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceb2d2ea9c474b06853d1891161296a8 in term 0.
I20260812 11:10:05.031178 29065 leader_election.cc:302] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 084ef8e47f8e4b6d8eac7275ef273315, ceb2d2ea9c474b06853d1891161296a8; no voters:
I20260812 11:10:05.031438 29379 raft_consensus.cc:2781] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:10:05.031499 29379 raft_consensus.cc:482] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:10:05.031599 29379 raft_consensus.cc:3037] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:05.032615 29379 raft_consensus.cc:504] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "084ef8e47f8e4b6d8eac7275ef273315" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 32917 } } peers { permanent_uuid: "ceb2d2ea9c474b06853d1891161296a8" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34889 } } peers { permanent_uuid: "9d36ca09efc643dd9d37dfedd9cd2403" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36081 } }
I20260812 11:10:05.033035 29379 leader_election.cc:288] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8 [CANDIDATE]: Term 1 election: Requested vote from peers 084ef8e47f8e4b6d8eac7275ef273315 (127.17.44.194:32917), 9d36ca09efc643dd9d37dfedd9cd2403 (127.17.44.193:36081)
I20260812 11:10:05.033385 28948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ddcf87fae354827a8ed1d5c9b052e88" candidate_uuid: "ceb2d2ea9c474b06853d1891161296a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "084ef8e47f8e4b6d8eac7275ef273315"
I20260812 11:10:05.033434 28746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ddcf87fae354827a8ed1d5c9b052e88" candidate_uuid: "ceb2d2ea9c474b06853d1891161296a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d36ca09efc643dd9d37dfedd9cd2403"
I20260812 11:10:05.033524 28948 raft_consensus.cc:3037] T 0ddcf87fae354827a8ed1d5c9b052e88 P 084ef8e47f8e4b6d8eac7275ef273315 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:05.033540 28746 raft_consensus.cc:3037] T 0ddcf87fae354827a8ed1d5c9b052e88 P 9d36ca09efc643dd9d37dfedd9cd2403 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:05.034797 28948 raft_consensus.cc:2454] T 0ddcf87fae354827a8ed1d5c9b052e88 P 084ef8e47f8e4b6d8eac7275ef273315 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb2d2ea9c474b06853d1891161296a8 in term 1.
I20260812 11:10:05.034797 28746 raft_consensus.cc:2454] T 0ddcf87fae354827a8ed1d5c9b052e88 P 9d36ca09efc643dd9d37dfedd9cd2403 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceb2d2ea9c474b06853d1891161296a8 in term 1.
I20260812 11:10:05.035142 29065 leader_election.cc:302] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 084ef8e47f8e4b6d8eac7275ef273315, ceb2d2ea9c474b06853d1891161296a8; no voters:
I20260812 11:10:05.035414 29379 raft_consensus.cc:2781] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:10:05.035681 29379 raft_consensus.cc:686] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8 [term 1 LEADER]: Becoming Leader. State: Replica: ceb2d2ea9c474b06853d1891161296a8, State: Running, Role: LEADER
I20260812 11:10:05.036104 29379 consensus_queue.cc:237] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "084ef8e47f8e4b6d8eac7275ef273315" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 32917 } } peers { permanent_uuid: "ceb2d2ea9c474b06853d1891161296a8" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34889 } } peers { permanent_uuid: "9d36ca09efc643dd9d37dfedd9cd2403" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36081 } }
W20260812 11:10:05.036490 28825 tablet.cc:2369] T 0ddcf87fae354827a8ed1d5c9b052e88 P 9d36ca09efc643dd9d37dfedd9cd2403: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:10:05.037734 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:05.036246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.195:60405)
0812 11:10:05.037363 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:05.037504 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":890}
I20260812 11:10:05.038179 28275 catalog_manager.cc:5495] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ceb2d2ea9c474b06853d1891161296a8 (127.17.44.195). New cstate: current_term: 1 leader_uuid: "ceb2d2ea9c474b06853d1891161296a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "084ef8e47f8e4b6d8eac7275ef273315" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 32917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb2d2ea9c474b06853d1891161296a8" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d36ca09efc643dd9d37dfedd9cd2403" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36081 } health_report { overall_health: UNKNOWN } } }
W20260812 11:10:05.045089 29030 tablet.cc:2369] T 0ddcf87fae354827a8ed1d5c9b052e88 P 084ef8e47f8e4b6d8eac7275ef273315: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:10:05.096385 29365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:05.092793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35646 (local address 127.17.44.195:34889)
0812 11:10:05.092959 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:05.092964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:05.092981 (+ 17us) server_negotiation.cc:407] Connection header received
0812 11:10:05.093036 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:05.093038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:05.093081 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:05.093163 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:10:05.093436 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:05.093940 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:05.094993 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:05.095300 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:05.095449 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:05.095792 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:10:05.095896 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:10:05.095983 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:05.096110 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 11:10:05.096185 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 11:10:05.099152 28746 raft_consensus.cc:1261] T 0ddcf87fae354827a8ed1d5c9b052e88 P 9d36ca09efc643dd9d37dfedd9cd2403 [term 1 FOLLOWER]: Refusing update from remote peer ceb2d2ea9c474b06853d1891161296a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:10:05.099582 28948 raft_consensus.cc:1261] T 0ddcf87fae354827a8ed1d5c9b052e88 P 084ef8e47f8e4b6d8eac7275ef273315 [term 1 FOLLOWER]: Refusing update from remote peer ceb2d2ea9c474b06853d1891161296a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:10:05.099738 29379 consensus_queue.cc:1035] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d36ca09efc643dd9d37dfedd9cd2403" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:10:05.100113 29388 consensus_queue.cc:1035] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "084ef8e47f8e4b6d8eac7275ef273315" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 32917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:10:05.102242 29385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:05.101898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.194:59045)
0812 11:10:05.102071 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:05.102142 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260812 11:10:05.102373 29386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:05.101679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.193:54223)
0812 11:10:05.102233 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:05.102293 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475}
I20260812 11:10:05.104626 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:05.104340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.195:50545)
0812 11:10:05.104495 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:05.104552 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260812 11:10:05.105808 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:05.105559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.195:57643)
0812 11:10:05.105691 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:05.105739 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260812 11:10:05.106324 29385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:05.106045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.194:58005)
0812 11:10:05.106222 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:05.106274 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260812 11:10:05.107815 29386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:05.107510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33165 (local address 127.17.44.193:50279)
0812 11:10:05.107702 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:05.107753 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260812 11:10:05.109884 28275 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:10:05.110073 28275 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:10:05.114706 28275 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 395bb06cf1e84d9ba73408995fc72721: Sending DeleteTablet for 3 replicas of tablet 0ddcf87fae354827a8ed1d5c9b052e88
I20260812 11:10:05.115466 29132 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ddcf87fae354827a8ed1d5c9b052e88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:10:05 UTC) from {username='oryx', principal='oryx/127.17.44.253@KRBTEST.COM'} at 127.0.0.1:35640
I20260812 11:10:05.115527 28923 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ddcf87fae354827a8ed1d5c9b052e88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:10:05 UTC) from {username='oryx', principal='oryx/127.17.44.253@KRBTEST.COM'} at 127.0.0.1:38714
I20260812 11:10:05.115654 28716 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ddcf87fae354827a8ed1d5c9b052e88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:10:05 UTC) from {username='oryx', principal='oryx/127.17.44.253@KRBTEST.COM'} at 127.0.0.1:42590
I20260812 11:10:05.115808 29407 tablet_replica.cc:330] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8: stopping tablet replica
I20260812 11:10:05.115854 29408 tablet_replica.cc:330] T 0ddcf87fae354827a8ed1d5c9b052e88 P 084ef8e47f8e4b6d8eac7275ef273315: stopping tablet replica
I20260812 11:10:05.115958 29407 raft_consensus.cc:2229] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:10:05.115978 29408 raft_consensus.cc:2229] T 0ddcf87fae354827a8ed1d5c9b052e88 P 084ef8e47f8e4b6d8eac7275ef273315 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:10:05.116113 29408 raft_consensus.cc:2258] T 0ddcf87fae354827a8ed1d5c9b052e88 P 084ef8e47f8e4b6d8eac7275ef273315 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:10:05.116178 29407 raft_consensus.cc:2258] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:10:05.116902 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 28492
I20260812 11:10:05.127493 29407 ts_tablet_manager.cc:1905] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:10:05.127555 29408 ts_tablet_manager.cc:1905] T 0ddcf87fae354827a8ed1d5c9b052e88 P 084ef8e47f8e4b6d8eac7275ef273315: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:10:05.129859 29407 ts_tablet_manager.cc:1918] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:10:05.129871 29408 ts_tablet_manager.cc:1918] T 0ddcf87fae354827a8ed1d5c9b052e88 P 084ef8e47f8e4b6d8eac7275ef273315: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:10:05.129961 29407 log.cc:1196] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/0ddcf87fae354827a8ed1d5c9b052e88
I20260812 11:10:05.129956 29408 log.cc:1196] T 0ddcf87fae354827a8ed1d5c9b052e88 P 084ef8e47f8e4b6d8eac7275ef273315: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/0ddcf87fae354827a8ed1d5c9b052e88
I20260812 11:10:05.130256 29407 ts_tablet_manager.cc:1939] T 0ddcf87fae354827a8ed1d5c9b052e88 P ceb2d2ea9c474b06853d1891161296a8: Deleting consensus metadata
I20260812 11:10:05.130260 29408 ts_tablet_manager.cc:1939] T 0ddcf87fae354827a8ed1d5c9b052e88 P 084ef8e47f8e4b6d8eac7275ef273315: Deleting consensus metadata
I20260812 11:10:05.131098 28257 catalog_manager.cc:4841] TS 084ef8e47f8e4b6d8eac7275ef273315 (127.17.44.194:32917): tablet 0ddcf87fae354827a8ed1d5c9b052e88 (table test-table [id=f9f89fe8061c46efbc5c0a8b83ec85ee]) successfully deleted
I20260812 11:10:05.131102 28258 catalog_manager.cc:4841] TS ceb2d2ea9c474b06853d1891161296a8 (127.17.44.195:34889): tablet 0ddcf87fae354827a8ed1d5c9b052e88 (table test-table [id=f9f89fe8061c46efbc5c0a8b83ec85ee]) successfully deleted
I20260812 11:10:05.137990 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 28834
I20260812 11:10:05.140552 29366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:05.138266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:36081 (local address 127.0.0.1:42592)
0812 11:10:05.140414 (+ 2148us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:05.140473 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.193:36081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2047}
W20260812 11:10:05.143591 28257 catalog_manager.cc:4584] TS 9d36ca09efc643dd9d37dfedd9cd2403 (127.17.44.193:36081): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0ddcf87fae354827a8ed1d5c9b052e88: Network error: Client connection negotiation failed: client connection to 127.17.44.193:36081: connect: Connection refused (error 111)
I20260812 11:10:05.151319 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 29035
I20260812 11:10:05.170746 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 28088
I20260812 11:10:05.182245 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 28235
2026-08-12T11:10:05Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (8106 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:10:05 dist-test-slave-drl0 krb5kdc[29413](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:10:05 dist-test-slave-drl0 krb5kdc[29413](info): set up 2 sockets
Aug 12 11:10:05 dist-test-slave-drl0 krb5kdc[29413](info): commencing operation
krb5kdc: starting...
W20260812 11:10:06.498337 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:10:06 dist-test-slave-drl0 krb5kdc[29413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:10:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:10:06Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.254.keytab.
Entry for principal oryx/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.254.keytab.
I20260812 11:10:06.643111 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:38053
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.254.keytab
--principal=oryx/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:35191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:38053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:10:06.764070 29433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:06.764359 29433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:06.764427 29433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:06.768388 29433 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:10:06.768465 29433 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 11:10:06.768496 29433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:06.768536 29433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:06.768569 29433 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:10:06.768604 29433 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:10:06.773821 29433 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:35191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:38053
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.254.keytab
--principal=oryx/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:38053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:06.774865 29433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:06.775903 29433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:06.785339 29440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:06.785504 29433 server_base.cc:1034] running on GCE node
W20260812 11:10:06.785356 29449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:06.785408 29445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:06.786185 29433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:06.787297 29433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:06.788506 29433 hybrid_clock.cc:648] HybridClock initialized: now 1786533006788487 us; error 31 us; skew 500 ppm
Aug 12 11:10:06 dist-test-slave-drl0 krb5kdc[29413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533006, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:10:06.796233 29433 init.cc:377] Logged in from keytab as oryx/127.17.44.254@KRBTEST.COM (short username oryx)
I20260812 11:10:06.797185 29433 webserver.cc:460] Webserver started at http://127.17.44.254:36439/ using document root <none> and password file <none>
I20260812 11:10:06.797783 29433 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:06.797853 29433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:06.798077 29433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:10:06.800463 29433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "977cbb4bc25242a8965c95da8803335c"
format_stamp: "Formatted at 2026-08-12 11:10:06 on dist-test-slave-drl0"
server_key: "20ee3cff7960371bf31b834ed9958f1a"
server_key_iv: "56fd083058a7356d4ba8d66d9b4df8c1"
server_key_version: "encryptionkey@0"
I20260812 11:10:06.801551 29433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "977cbb4bc25242a8965c95da8803335c"
format_stamp: "Formatted at 2026-08-12 11:10:06 on dist-test-slave-drl0"
server_key: "20ee3cff7960371bf31b834ed9958f1a"
server_key_iv: "56fd083058a7356d4ba8d66d9b4df8c1"
server_key_version: "encryptionkey@0"
I20260812 11:10:06.805804 29433 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 11:10:06.808832 29458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:06.810338 29433 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:10:06.810501 29433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "977cbb4bc25242a8965c95da8803335c"
format_stamp: "Formatted at 2026-08-12 11:10:06 on dist-test-slave-drl0"
server_key: "20ee3cff7960371bf31b834ed9958f1a"
server_key_iv: "56fd083058a7356d4ba8d66d9b4df8c1"
server_key_version: "encryptionkey@0"
I20260812 11:10:06.810629 29433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:06.823356 29433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:06.827809 29433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:06.828042 29433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:06.837599 29433 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:38053
I20260812 11:10:06.837669 29554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:38053 every 8 connection(s)
I20260812 11:10:06.838621 29433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:10:06.841183 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 29433
I20260812 11:10:06.841383 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:10:06.841694 29555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:06.841846 17587 external_mini_cluster.cc:1428] Setting key 0ac416d5534a1d31d931a964f3bfa530
I20260812 11:10:06.848943 29555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c: Bootstrap starting.
Aug 12 11:10:06 dist-test-slave-drl0 krb5kdc[29413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM
I20260812 11:10:06.852759 29555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:06.853523 29555 log.cc:824] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c: Log is configured to *not* fsync() on all Append() calls
I20260812 11:10:06.855803 29555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c: No bootstrap required, opened a new log
I20260812 11:10:06.859301 29555 raft_consensus.cc:348] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "977cbb4bc25242a8965c95da8803335c" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 38053 } }
I20260812 11:10:06.859529 29555 raft_consensus.cc:374] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:06.859565 29555 raft_consensus.cc:729] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977cbb4bc25242a8965c95da8803335c, State: Initialized, Role: FOLLOWER
I20260812 11:10:06.860193 29555 consensus_queue.cc:260] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "977cbb4bc25242a8965c95da8803335c" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 38053 } }
I20260812 11:10:06.860510 29555 raft_consensus.cc:388] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:10:06.860610 29555 raft_consensus.cc:482] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:10:06.860749 29555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:06.860865 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:06.843781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39380 (local address 127.17.44.254:38053)
0812 11:10:06.844234 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:06.844259 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:06.844295 (+ 36us) server_negotiation.cc:407] Connection header received
0812 11:10:06.845002 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:06.845019 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:06.845330 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:06.845605 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:06.846690 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:06.847888 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:06.849115 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:06.849586 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:06.849710 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:06.853192 (+ 3482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:06.853197 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:06.853210 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:06.853238 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:06.855797 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:06.856383 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:06.856389 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:06.856394 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:06.856509 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:06.856945 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:06.856950 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:06.856952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:06.857688 (+ 736us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:06.857938 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:06.858285 (+ 347us) server_negotiation.cc:299] Negotiation successful
0812 11:10:06.858643 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":173,"threads_started":1}
I20260812 11:10:06.862134 29555 raft_consensus.cc:504] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "977cbb4bc25242a8965c95da8803335c" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 38053 } }
I20260812 11:10:06.862638 29555 leader_election.cc:302] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 977cbb4bc25242a8965c95da8803335c; no voters:
I20260812 11:10:06.863219 29555 leader_election.cc:288] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:10:06.863288 29563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:10:06.863615 29563 raft_consensus.cc:686] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c [term 1 LEADER]: Becoming Leader. State: Replica: 977cbb4bc25242a8965c95da8803335c, State: Running, Role: LEADER
I20260812 11:10:06.863986 29563 consensus_queue.cc:237] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "977cbb4bc25242a8965c95da8803335c" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 38053 } }
I20260812 11:10:06.864831 29555 sys_catalog.cc:564] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:10:06.865172 29565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 977cbb4bc25242a8965c95da8803335c. Latest consensus state: current_term: 1 leader_uuid: "977cbb4bc25242a8965c95da8803335c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "977cbb4bc25242a8965c95da8803335c" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 38053 } } }
I20260812 11:10:06.865332 29565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c [sys.catalog]: This master's current role is: LEADER
I20260812 11:10:06.865625 29564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "977cbb4bc25242a8965c95da8803335c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "977cbb4bc25242a8965c95da8803335c" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 38053 } } }
I20260812 11:10:06.865746 29564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c [sys.catalog]: This master's current role is: LEADER
I20260812 11:10:06.865795 29568 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:10:06.869154 29568 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:10:06.877058 29568 catalog_manager.cc:1347] Generated new cluster ID: 20ee4ab46a164ec0bbe8ac8352c49a18
I20260812 11:10:06.877161 29568 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:10:06.889578 29568 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:10:06.891176 29568 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:10:06.898468 29568 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c: Generated new TSK 0
I20260812 11:10:06.899312 29568 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.193.keytab.
Entry for principal oryx/127.17.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.193.keytab.
I20260812 11:10:06.931391 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:38053
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.193.keytab
--principal=oryx/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:35191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:10:07.056841 29593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:07.057103 29593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:07.057147 29593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:07.061358 29593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:07.061415 29593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:07.061455 29593 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 11:10:07.061487 29593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:10:07.067471 29593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:35191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.193.keytab
--principal=oryx/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.44.254:38053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:07.068531 29593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:07.069803 29593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:07.080128 29601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:07.080176 29602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:07.080176 29606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:07.081637 29593 server_base.cc:1034] running on GCE node
I20260812 11:10:07.082424 29593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:07.083422 29593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:07.084661 29593 hybrid_clock.cc:648] HybridClock initialized: now 1786533007084630 us; error 46 us; skew 500 ppm
Aug 12 11:10:07 dist-test-slave-drl0 krb5kdc[29413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533007, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:10:07.088090 29593 init.cc:377] Logged in from keytab as oryx/127.17.44.193@KRBTEST.COM (short username oryx)
I20260812 11:10:07.089260 29593 webserver.cc:460] Webserver started at http://127.17.44.193:44121/ using document root <none> and password file <none>
I20260812 11:10:07.089860 29593 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:07.089960 29593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:07.090312 29593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:10:07.092505 29593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "d7a6f415051b487eaefa063e0bb43960"
format_stamp: "Formatted at 2026-08-12 11:10:07 on dist-test-slave-drl0"
server_key: "5b7eba1b3048a7d296f24ee4c353a52c"
server_key_iv: "2f78ce3f2468949907fedf53252e0325"
server_key_version: "encryptionkey@0"
I20260812 11:10:07.093087 29593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "d7a6f415051b487eaefa063e0bb43960"
format_stamp: "Formatted at 2026-08-12 11:10:07 on dist-test-slave-drl0"
server_key: "5b7eba1b3048a7d296f24ee4c353a52c"
server_key_iv: "2f78ce3f2468949907fedf53252e0325"
server_key_version: "encryptionkey@0"
I20260812 11:10:07.096956 29593 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260812 11:10:07.100180 29618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:07.102133 29593 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260812 11:10:07.102306 29593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "d7a6f415051b487eaefa063e0bb43960"
format_stamp: "Formatted at 2026-08-12 11:10:07 on dist-test-slave-drl0"
server_key: "5b7eba1b3048a7d296f24ee4c353a52c"
server_key_iv: "2f78ce3f2468949907fedf53252e0325"
server_key_version: "encryptionkey@0"
I20260812 11:10:07.102428 29593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:07.115355 29593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:07.118562 29593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:07.118763 29593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:07.120555 29593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:10:07.120625 29593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:07.120684 29593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:10:07.120708 29593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 11:10:07 dist-test-slave-drl0 krb5kdc[29413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533007, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM
I20260812 11:10:07.132623 29593 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:39365
I20260812 11:10:07.132712 29800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:39365 every 8 connection(s)
I20260812 11:10:07.133493 29593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:10:07.135416 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:07.121375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:33979 (local address 127.17.44.254:38053)
0812 11:10:07.121581 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:07.121586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:07.122391 (+ 805us) server_negotiation.cc:407] Connection header received
0812 11:10:07.123405 (+ 1014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:07.123408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:07.123488 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:07.123608 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:07.124932 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.125634 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.127179 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.127430 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.127548 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.131282 (+ 3734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:07.131287 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:07.131290 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:07.131320 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:07.133049 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:07.133625 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:07.133629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:07.133631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:07.133713 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:07.134240 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:07.134244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:07.134245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:07.134490 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:07.134578 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:07.135107 (+ 529us) server_negotiation.cc:299] Negotiation successful
0812 11:10:07.135242 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 11:10:07.136132 29629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:07.121659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:38053 (local address 127.17.44.193:33979)
0812 11:10:07.122200 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:07.122254 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:07.123198 (+ 944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:07.123781 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:07.123789 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:07.124284 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:07.124776 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:07.124795 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.125763 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.125767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:07.127053 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:07.127064 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.127595 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.127599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:07.127776 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.128354 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:07.128380 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:07.131111 (+ 2731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:07.133190 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:07.133197 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:07.133213 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:07.133482 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:07.133810 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:07.133815 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:07.133821 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:07.134097 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:07.134604 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:07.134609 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:07.134818 (+ 209us) client_negotiation.cc:769] Sending connection context
0812 11:10:07.135029 (+ 211us) client_negotiation.cc:240] Negotiation successful
0812 11:10:07.135281 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":151,"threads_started":1}
I20260812 11:10:07.137954 29801 heartbeater.cc:347] Connected to a master server at 127.17.44.254:38053
I20260812 11:10:07.138103 29801 heartbeater.cc:464] Registering TS with master...
I20260812 11:10:07.138623 29801 heartbeater.cc:511] Master 127.17.44.254:38053 requested a full tablet report, sending...
I20260812 11:10:07.140172 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 29593
I20260812 11:10:07.140317 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:10:07.140389 29478 ts_manager.cc:194] Registered new tserver with Master: d7a6f415051b487eaefa063e0bb43960 (127.17.44.193:39365)
I20260812 11:10:07.140647 17587 external_mini_cluster.cc:1428] Setting key 715490311a628df8bcd864cee9798f06
I20260812 11:10:07.141947 29478 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.44.193@KRBTEST.COM'} at 127.17.44.193:33979
I20260812 11:10:07.150264 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:07.141184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:33187 (local address 127.17.44.254:38053)
0812 11:10:07.141299 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:07.141303 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:07.141315 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:10:07.141352 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:07.141355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:07.141388 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:07.141477 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:07.141919 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.142525 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.143723 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.144117 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.144238 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.145467 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:07.145470 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:07.145472 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:07.145494 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:07.147697 (+ 2203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:07.148169 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:07.148175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:07.148180 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:07.148292 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:07.148712 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:07.148720 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:07.148725 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:07.149242 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:07.149421 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:07.150045 (+ 624us) server_negotiation.cc:299] Negotiation successful
0812 11:10:07.150152 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 11:10:07.150519 29809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:07.140813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:38053 (local address 127.17.44.193:33187)
0812 11:10:07.141101 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:07.141117 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:07.141210 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:07.141496 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:07.141499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:07.141700 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:07.141792 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:07.141798 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.142683 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.142686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:07.143568 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:07.143579 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.144254 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.144258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:07.144353 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.144803 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:07.144822 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:07.145302 (+ 480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:07.147853 (+ 2551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:07.147855 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:07.147857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:07.148055 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:07.148429 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:07.148431 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:07.148433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:07.148519 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:07.149425 (+ 906us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:07.149428 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:07.149707 (+ 279us) client_negotiation.cc:769] Sending connection context
0812 11:10:07.149903 (+ 196us) client_negotiation.cc:240] Negotiation successful
0812 11:10:07.150199 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":125,"threads_started":1}
WARNING: no policy specified for oryx/127.17.44.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.44.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.194.keytab.
Entry for principal oryx/127.17.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.194.keytab.
I20260812 11:10:07.178115 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:38053
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.194.keytab
--principal=oryx/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:35191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:10:07.304270 29814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:07.304549 29814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:07.304593 29814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:07.308959 29814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:07.309023 29814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:07.309063 29814 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 11:10:07.309096 29814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:10:07.314517 29814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:35191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.194.keytab
--principal=oryx/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.44.254:38053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:07.315719 29814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:07.316838 29814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:07.327683 29822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:07.327836 29819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:07.327956 29814 server_base.cc:1034] running on GCE node
W20260812 11:10:07.327685 29824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:07.328871 29814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:07.329548 29814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:07.330819 29814 hybrid_clock.cc:648] HybridClock initialized: now 1786533007330773 us; error 57 us; skew 500 ppm
Aug 12 11:10:07 dist-test-slave-drl0 krb5kdc[29413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533007, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:10:07.333772 29814 init.cc:377] Logged in from keytab as oryx/127.17.44.194@KRBTEST.COM (short username oryx)
I20260812 11:10:07.334630 29814 webserver.cc:460] Webserver started at http://127.17.44.194:37823/ using document root <none> and password file <none>
I20260812 11:10:07.335280 29814 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:07.335373 29814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:07.335613 29814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:10:07.337680 29814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "03be7361f2d54061897ab2be44de9a2d"
format_stamp: "Formatted at 2026-08-12 11:10:07 on dist-test-slave-drl0"
server_key: "bfd1aceedbf6c688c3f687e3caa4735e"
server_key_iv: "2f2802ffbb5c162ae1ab5ef017df0f2f"
server_key_version: "encryptionkey@0"
I20260812 11:10:07.338277 29814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "03be7361f2d54061897ab2be44de9a2d"
format_stamp: "Formatted at 2026-08-12 11:10:07 on dist-test-slave-drl0"
server_key: "bfd1aceedbf6c688c3f687e3caa4735e"
server_key_iv: "2f2802ffbb5c162ae1ab5ef017df0f2f"
server_key_version: "encryptionkey@0"
I20260812 11:10:07.342463 29814 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 11:10:07.345788 29834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:07.347577 29814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 11:10:07.347751 29814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "03be7361f2d54061897ab2be44de9a2d"
format_stamp: "Formatted at 2026-08-12 11:10:07 on dist-test-slave-drl0"
server_key: "bfd1aceedbf6c688c3f687e3caa4735e"
server_key_iv: "2f2802ffbb5c162ae1ab5ef017df0f2f"
server_key_version: "encryptionkey@0"
I20260812 11:10:07.347885 29814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:07.364250 29814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:07.368297 29814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:07.368476 29814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:07.370147 29814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:10:07.370234 29814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:07.370404 29814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:10:07.370469 29814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 11:10:07 dist-test-slave-drl0 krb5kdc[29413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533007, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM
I20260812 11:10:07.382395 29814 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:43485
I20260812 11:10:07.382472 30009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:43485 every 8 connection(s)
I20260812 11:10:07.383625 29814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:10:07.386132 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 29814
I20260812 11:10:07.386170 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:07.371739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:33781 (local address 127.17.44.254:38053)
0812 11:10:07.371877 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:07.371888 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:07.373017 (+ 1129us) server_negotiation.cc:407] Connection header received
0812 11:10:07.374496 (+ 1479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:07.374501 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:07.374570 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:07.374691 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:07.376226 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.377092 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.378914 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.379293 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.379415 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.382242 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:07.382246 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:07.382249 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:07.382273 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:07.383930 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:07.384588 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:07.384594 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:07.384596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:07.384700 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:07.385058 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:07.385062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:07.385063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:07.385254 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:07.385369 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:07.385862 (+ 493us) server_negotiation.cc:299] Negotiation successful
0812 11:10:07.385999 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:10:07.386300 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:10:07.386726 17587 external_mini_cluster.cc:1428] Setting key 95fb86c4f1dceca2e9dcadc9e08e5974
I20260812 11:10:07.387091 29848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:07.372249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:38053 (local address 127.17.44.194:33781)
0812 11:10:07.372830 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:07.372875 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:07.374213 (+ 1338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:07.375022 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:07.375034 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:07.375435 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:07.376075 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:07.376095 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.377234 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.377238 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:07.378771 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:07.378782 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.379441 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.379444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:07.379578 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.380173 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:07.380194 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:07.382043 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:07.384067 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:07.384074 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:07.384090 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:07.384466 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:07.384801 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:07.384804 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:07.384806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:07.384951 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:07.385367 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:07.385384 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:07.385620 (+ 236us) client_negotiation.cc:769] Sending connection context
0812 11:10:07.385813 (+ 193us) client_negotiation.cc:240] Negotiation successful
0812 11:10:07.386148 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":152,"threads_started":1}
I20260812 11:10:07.388675 30010 heartbeater.cc:347] Connected to a master server at 127.17.44.254:38053
I20260812 11:10:07.388845 30010 heartbeater.cc:464] Registering TS with master...
I20260812 11:10:07.389344 30010 heartbeater.cc:511] Master 127.17.44.254:38053 requested a full tablet report, sending...
I20260812 11:10:07.390493 29478 ts_manager.cc:194] Registered new tserver with Master: 03be7361f2d54061897ab2be44de9a2d (127.17.44.194:43485)
I20260812 11:10:07.391330 29478 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.44.194@KRBTEST.COM'} at 127.17.44.194:33781
WARNING: no policy specified for oryx/127.17.44.195@KRBTEST.COM; defaulting to no policy
I20260812 11:10:07.398844 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:07.391233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:33041 (local address 127.17.44.254:38053)
0812 11:10:07.391333 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:07.391337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:07.391686 (+ 349us) server_negotiation.cc:407] Connection header received
0812 11:10:07.391857 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:07.391860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:07.391915 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:07.391997 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:07.392472 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.392979 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.393824 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.394056 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.394165 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.395353 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:07.395357 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:07.395359 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:07.395384 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:07.396818 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:07.397253 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:07.397258 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:07.397259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:07.397340 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:07.397696 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:07.397700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:07.397701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:07.398013 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:07.398160 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:07.398561 (+ 401us) server_negotiation.cc:299] Negotiation successful
0812 11:10:07.398722 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 11:10:07.398913 30024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:07.391182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:38053 (local address 127.17.44.194:33041)
0812 11:10:07.391611 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:07.391627 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:07.391755 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:07.392015 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:07.392018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:07.392217 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:07.392345 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:07.392351 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.393109 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.393113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:07.393692 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:07.393701 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.394171 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.394175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:07.394267 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.394806 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:07.394817 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:07.395206 (+ 389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:07.396941 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:07.396947 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:07.396949 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:07.397150 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:07.397437 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:07.397441 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:07.397443 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:07.397556 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:07.398219 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:07.398226 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:07.398348 (+ 122us) client_negotiation.cc:769] Sending connection context
0812 11:10:07.398494 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 11:10:07.398663 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":8,"thread_start_us":220,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.195.keytab.
Entry for principal oryx/127.17.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.195.keytab.
I20260812 11:10:07.426638 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:38053
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.195.keytab
--principal=oryx/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:35191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:10:07.559465 30026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:07.559713 30026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:07.559758 30026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:07.564342 30026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:07.564432 30026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:07.564504 30026 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 11:10:07.564594 30026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:10:07.570291 30026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:35191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.195.keytab
--principal=oryx/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.44.254:38053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:07.571460 30026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:07.572513 30026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:07.582103 30037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:07.582118 30034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:07.582109 30035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:07.582547 30026 server_base.cc:1034] running on GCE node
I20260812 11:10:07.582965 30026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:07.583580 30026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:07.584858 30026 hybrid_clock.cc:648] HybridClock initialized: now 1786533007584825 us; error 50 us; skew 500 ppm
Aug 12 11:10:07 dist-test-slave-drl0 krb5kdc[29413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533007, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:10:07.587913 30026 init.cc:377] Logged in from keytab as oryx/127.17.44.195@KRBTEST.COM (short username oryx)
I20260812 11:10:07.589038 30026 webserver.cc:460] Webserver started at http://127.17.44.195:43297/ using document root <none> and password file <none>
I20260812 11:10:07.589665 30026 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:07.589732 30026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:07.589953 30026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:10:07.592635 30026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "82875c708dc345eba57b0ea8d0a4dc3f"
format_stamp: "Formatted at 2026-08-12 11:10:07 on dist-test-slave-drl0"
server_key: "1e703c29cdef590836e6312138ea1613"
server_key_iv: "5d89b5da3014029a9e48e68e113ebd00"
server_key_version: "encryptionkey@0"
I20260812 11:10:07.593612 30026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "82875c708dc345eba57b0ea8d0a4dc3f"
format_stamp: "Formatted at 2026-08-12 11:10:07 on dist-test-slave-drl0"
server_key: "1e703c29cdef590836e6312138ea1613"
server_key_iv: "5d89b5da3014029a9e48e68e113ebd00"
server_key_version: "encryptionkey@0"
I20260812 11:10:07.598379 30026 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 11:10:07.601110 30051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:07.602423 30026 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:10:07.602546 30026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "82875c708dc345eba57b0ea8d0a4dc3f"
format_stamp: "Formatted at 2026-08-12 11:10:07 on dist-test-slave-drl0"
server_key: "1e703c29cdef590836e6312138ea1613"
server_key_iv: "5d89b5da3014029a9e48e68e113ebd00"
server_key_version: "encryptionkey@0"
I20260812 11:10:07.602672 30026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:07.633563 30026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:07.637075 30026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:07.637238 30026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:07.638986 30026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:10:07.639074 30026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:07.639132 30026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:10:07.639168 30026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:07.650799 30026 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:46469
I20260812 11:10:07.651074 30228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:46469 every 8 connection(s)
Aug 12 11:10:07 dist-test-slave-drl0 krb5kdc[29413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533007, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM
I20260812 11:10:07.651983 30026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:10:07.656219 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 30026
I20260812 11:10:07.656370 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:10:07.656741 17587 external_mini_cluster.cc:1428] Setting key 345a1603e7c573221ccc1b0b12c03c39
I20260812 11:10:07.657966 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:07.640661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:46815 (local address 127.17.44.254:38053)
0812 11:10:07.640872 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:07.640879 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:07.642128 (+ 1249us) server_negotiation.cc:407] Connection header received
0812 11:10:07.643816 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:07.643826 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:07.643895 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:07.644019 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:07.645890 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.646550 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.648345 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.648834 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.648971 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.652539 (+ 3568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:07.652544 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:07.652548 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:07.652605 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:07.654809 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:07.655682 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:07.655685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:07.655687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:07.655774 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:07.656230 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:07.656235 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:07.656238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:07.656492 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:07.656613 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:07.657567 (+ 954us) server_negotiation.cc:299] Negotiation successful
0812 11:10:07.657799 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 11:10:07.658811 30069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:07.641271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:38053 (local address 127.17.44.195:46815)
0812 11:10:07.641908 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:07.641980 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:07.643458 (+ 1478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:07.644260 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:07.644271 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:07.644830 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:07.645612 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:07.645637 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.646667 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.646672 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:07.648183 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:07.648195 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.649187 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.649193 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:07.649451 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.650094 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:07.650117 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:07.652221 (+ 2104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:07.655088 (+ 2867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:07.655096 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:07.655114 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:07.655513 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:07.655907 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:07.655911 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:07.655913 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:07.656078 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:07.656662 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:07.656669 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:07.657203 (+ 534us) client_negotiation.cc:769] Sending connection context
0812 11:10:07.657447 (+ 244us) client_negotiation.cc:240] Negotiation successful
0812 11:10:07.657801 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":192,"threads_started":1}
I20260812 11:10:07.660046 30230 heartbeater.cc:347] Connected to a master server at 127.17.44.254:38053
I20260812 11:10:07.660336 30230 heartbeater.cc:464] Registering TS with master...
I20260812 11:10:07.661294 30230 heartbeater.cc:511] Master 127.17.44.254:38053 requested a full tablet report, sending...
I20260812 11:10:07.662738 29484 ts_manager.cc:194] Registered new tserver with Master: 82875c708dc345eba57b0ea8d0a4dc3f (127.17.44.195:46469)
I20260812 11:10:07.663465 29484 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.44.195@KRBTEST.COM'} at 127.17.44.195:46815
I20260812 11:10:07.671460 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:10:07.671602 30236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:07.662576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:38053 (local address 127.17.44.195:43993)
0812 11:10:07.663036 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:07.663055 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:07.663159 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:07.663426 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:07.663429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:07.663670 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:07.663803 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:07.663810 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.665290 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.665294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:07.665972 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:07.665981 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.666588 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.666591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:07.666669 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.667364 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:07.667389 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:07.667711 (+ 322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:07.669316 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:07.669319 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:07.669321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:07.669518 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:07.669984 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:07.669991 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:07.669996 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:07.670230 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:07.670830 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:07.670834 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:07.671023 (+ 189us) client_negotiation.cc:769] Sending connection context
0812 11:10:07.671159 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 11:10:07.671371 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":130,"threads_started":1}
I20260812 11:10:07.671602 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:07.662993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:43993 (local address 127.17.44.254:38053)
0812 11:10:07.663141 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:07.663145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:07.663159 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:10:07.663255 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:07.663258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:07.663308 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:07.663399 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:07.664000 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.664888 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.666096 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.666423 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.666525 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.667869 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:07.667872 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:07.667874 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:07.667895 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:07.669164 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:07.669634 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:07.669639 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:07.669641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:07.669755 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:07.670367 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:07.670371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:07.670373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:07.670691 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:07.670844 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:07.671221 (+ 377us) server_negotiation.cc:299] Negotiation successful
0812 11:10:07.671366 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Aug 12 11:10:07 dist-test-slave-drl0 krb5kdc[29413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533007, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 11:10:07 dist-test-slave-drl0 krb5kdc[29413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:07 dist-test-slave-drl0 krb5kdc[29413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 11:10:07.697952 29559 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:07.690997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39388 (local address 127.17.44.254:38053)
0812 11:10:07.691131 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:07.691135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:07.691148 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:10:07.691188 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:07.691190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:07.691228 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:07.691307 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:07.692429 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.692978 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.693938 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.694291 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.694433 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.697790 (+ 3357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39388: BlockingRecv error: recv got EOF from 127.0.0.1:39388 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 12 11:10:07 dist-test-slave-drl0 krb5kdc[29413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533007, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM
I20260812 11:10:07.710636 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:07.699952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39392 (local address 127.17.44.254:38053)
0812 11:10:07.700112 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:07.700116 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:07.700254 (+ 138us) server_negotiation.cc:407] Connection header received
0812 11:10:07.700421 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:07.700425 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:07.700474 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:07.700559 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:07.701157 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.701749 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.702569 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.702784 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.702894 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.705987 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:07.705992 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:07.705994 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:07.706033 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:07.708530 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:07.709081 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:07.709086 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:07.709087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:07.709180 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:07.709506 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:07.709511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:07.709513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:07.709746 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:07.709914 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:07.710316 (+ 402us) server_negotiation.cc:299] Negotiation successful
0812 11:10:07.710455 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 11:10:07.714259 29484 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:10:07.717993 29484 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:10:07.733891 30256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:07.727813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:43485 (local address 127.0.0.1:53518)
0812 11:10:07.728569 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:07.728606 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:07.728775 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:07.729713 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:07.729720 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:07.729750 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:07.729896 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:07.729905 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.731275 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.731281 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:07.732449 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:07.732459 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.733248 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.733252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:07.733361 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.733399 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 11:10:07.733535 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 11:10:07.733660 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":257,"threads_started":1}
I20260812 11:10:07.733906 30258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:07.728019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53518 (local address 127.17.44.194:43485)
0812 11:10:07.728386 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:07.728400 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:07.728735 (+ 335us) server_negotiation.cc:407] Connection header received
0812 11:10:07.728983 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:07.728992 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:07.729346 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:07.729516 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:07.730038 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.731158 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.732581 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.733121 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.733246 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.733412 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:07.733566 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 11:10:07.733660 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":145,"threads_started":1}
I20260812 11:10:07.734973 30259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:07.729037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:46469 (local address 127.0.0.1:42292)
0812 11:10:07.729764 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:07.729779 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:07.729947 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:07.730400 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:07.730403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:07.730421 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:07.730530 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:07.730536 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.732182 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.732185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:07.733290 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:07.733299 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.734545 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.734548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:07.734648 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.734669 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 11:10:07.734732 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 11:10:07.734809 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":116,"threads_started":1}
I20260812 11:10:07.735411 30261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:07.729224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42292 (local address 127.17.44.195:46469)
0812 11:10:07.729737 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:07.729748 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:07.729868 (+ 120us) server_negotiation.cc:407] Connection header received
0812 11:10:07.730123 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:07.730128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:07.730263 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:07.730419 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:07.730662 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.732070 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.733420 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.734213 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.734295 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.734439 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:07.735156 (+ 717us) server_negotiation.cc:299] Negotiation successful
0812 11:10:07.735264 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":134,"threads_started":1}
I20260812 11:10:07.735423 30262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:07.729468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:39365 (local address 127.0.0.1:55958)
0812 11:10:07.729876 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:07.729889 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:07.729961 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:07.730585 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:07.730588 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:07.730605 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:07.730705 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:07.730711 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.732041 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.732045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:07.733247 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:07.733267 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.734858 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.734861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:07.734971 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.735003 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 11:10:07.735241 (+ 238us) client_negotiation.cc:240] Negotiation successful
0812 11:10:07.735319 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":95,"threads_started":1}
I20260812 11:10:07.735780 30263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:07.729632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55958 (local address 127.17.44.193:39365)
0812 11:10:07.730038 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:07.730045 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:07.730065 (+ 20us) server_negotiation.cc:407] Connection header received
0812 11:10:07.730141 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:07.730146 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:07.730313 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:07.730452 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:07.730837 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.731913 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.733408 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.734553 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.734644 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.734799 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:07.735585 (+ 786us) server_negotiation.cc:299] Negotiation successful
0812 11:10:07.735676 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":97,"threads_started":1}
I20260812 11:10:07.736963 29915 tablet_service.cc:1467] Processing CreateTablet for tablet 21896524823f412fbd1bd5249cc72a7a (DEFAULT_TABLE table=test-table [id=4b7004b6aefc4677aea4b06fb26f0d5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:10:07.738143 29915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21896524823f412fbd1bd5249cc72a7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:07.740105 30124 tablet_service.cc:1467] Processing CreateTablet for tablet 21896524823f412fbd1bd5249cc72a7a (DEFAULT_TABLE table=test-table [id=4b7004b6aefc4677aea4b06fb26f0d5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:10:07.740931 30124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21896524823f412fbd1bd5249cc72a7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:07.741138 29700 tablet_service.cc:1467] Processing CreateTablet for tablet 21896524823f412fbd1bd5249cc72a7a (DEFAULT_TABLE table=test-table [id=4b7004b6aefc4677aea4b06fb26f0d5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:10:07.742249 29700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21896524823f412fbd1bd5249cc72a7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:07.747840 30264 tablet_bootstrap.cc:492] T 21896524823f412fbd1bd5249cc72a7a P 82875c708dc345eba57b0ea8d0a4dc3f: Bootstrap starting.
I20260812 11:10:07.750149 30264 tablet_bootstrap.cc:654] T 21896524823f412fbd1bd5249cc72a7a P 82875c708dc345eba57b0ea8d0a4dc3f: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:07.751195 30267 tablet_bootstrap.cc:492] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960: Bootstrap starting.
I20260812 11:10:07.751474 30264 log.cc:824] T 21896524823f412fbd1bd5249cc72a7a P 82875c708dc345eba57b0ea8d0a4dc3f: Log is configured to *not* fsync() on all Append() calls
I20260812 11:10:07.753636 30267 tablet_bootstrap.cc:654] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:07.753732 30264 tablet_bootstrap.cc:492] T 21896524823f412fbd1bd5249cc72a7a P 82875c708dc345eba57b0ea8d0a4dc3f: No bootstrap required, opened a new log
I20260812 11:10:07.753695 30270 tablet_bootstrap.cc:492] T 21896524823f412fbd1bd5249cc72a7a P 03be7361f2d54061897ab2be44de9a2d: Bootstrap starting.
I20260812 11:10:07.753954 30264 ts_tablet_manager.cc:1397] T 21896524823f412fbd1bd5249cc72a7a P 82875c708dc345eba57b0ea8d0a4dc3f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 11:10:07.754570 30267 log.cc:824] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960: Log is configured to *not* fsync() on all Append() calls
I20260812 11:10:07.756337 30270 tablet_bootstrap.cc:654] T 21896524823f412fbd1bd5249cc72a7a P 03be7361f2d54061897ab2be44de9a2d: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:07.757164 30270 log.cc:824] T 21896524823f412fbd1bd5249cc72a7a P 03be7361f2d54061897ab2be44de9a2d: Log is configured to *not* fsync() on all Append() calls
I20260812 11:10:07.757691 30267 tablet_bootstrap.cc:492] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960: No bootstrap required, opened a new log
I20260812 11:10:07.757889 30267 ts_tablet_manager.cc:1397] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 11:10:07.758878 30270 tablet_bootstrap.cc:492] T 21896524823f412fbd1bd5249cc72a7a P 03be7361f2d54061897ab2be44de9a2d: No bootstrap required, opened a new log
I20260812 11:10:07.759112 30270 ts_tablet_manager.cc:1397] T 21896524823f412fbd1bd5249cc72a7a P 03be7361f2d54061897ab2be44de9a2d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:10:07.758854 30264 raft_consensus.cc:348] T 21896524823f412fbd1bd5249cc72a7a P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } } peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } }
I20260812 11:10:07.759614 30264 raft_consensus.cc:374] T 21896524823f412fbd1bd5249cc72a7a P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:07.759714 30264 raft_consensus.cc:729] T 21896524823f412fbd1bd5249cc72a7a P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82875c708dc345eba57b0ea8d0a4dc3f, State: Initialized, Role: FOLLOWER
I20260812 11:10:07.760443 30264 consensus_queue.cc:260] T 21896524823f412fbd1bd5249cc72a7a P 82875c708dc345eba57b0ea8d0a4dc3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } } peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } }
I20260812 11:10:07.761435 30230 heartbeater.cc:503] Master 127.17.44.254:38053 was elected leader, sending a full tablet report...
I20260812 11:10:07.761879 30264 ts_tablet_manager.cc:1428] T 21896524823f412fbd1bd5249cc72a7a P 82875c708dc345eba57b0ea8d0a4dc3f: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260812 11:10:07.762871 30270 raft_consensus.cc:348] T 21896524823f412fbd1bd5249cc72a7a P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } } peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } }
I20260812 11:10:07.763206 30270 raft_consensus.cc:374] T 21896524823f412fbd1bd5249cc72a7a P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:07.763270 30270 raft_consensus.cc:729] T 21896524823f412fbd1bd5249cc72a7a P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03be7361f2d54061897ab2be44de9a2d, State: Initialized, Role: FOLLOWER
I20260812 11:10:07.763115 30267 raft_consensus.cc:348] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } } peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } }
I20260812 11:10:07.763407 30267 raft_consensus.cc:374] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:07.763494 30267 raft_consensus.cc:729] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7a6f415051b487eaefa063e0bb43960, State: Initialized, Role: FOLLOWER
I20260812 11:10:07.763792 30270 consensus_queue.cc:260] T 21896524823f412fbd1bd5249cc72a7a P 03be7361f2d54061897ab2be44de9a2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } } peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } }
I20260812 11:10:07.763908 30267 consensus_queue.cc:260] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } } peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } }
I20260812 11:10:07.764532 30010 heartbeater.cc:503] Master 127.17.44.254:38053 was elected leader, sending a full tablet report...
I20260812 11:10:07.764983 30267 ts_tablet_manager.cc:1428] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260812 11:10:07.765003 30270 ts_tablet_manager.cc:1428] T 21896524823f412fbd1bd5249cc72a7a P 03be7361f2d54061897ab2be44de9a2d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:10:07.765726 29801 heartbeater.cc:503] Master 127.17.44.254:38053 was elected leader, sending a full tablet report...
I20260812 11:10:07.798581 30280 raft_consensus.cc:482] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:10:07.798911 30280 raft_consensus.cc:504] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } } peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } }
I20260812 11:10:07.800220 30280 leader_election.cc:288] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03be7361f2d54061897ab2be44de9a2d (127.17.44.194:43485), 82875c708dc345eba57b0ea8d0a4dc3f (127.17.44.195:46469)
I20260812 11:10:07.804752 30258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:07.800687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:55289 (local address 127.17.44.194:43485)
0812 11:10:07.800892 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:07.800898 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:07.801070 (+ 172us) server_negotiation.cc:407] Connection header received
0812 11:10:07.801225 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:07.801228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:07.801285 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:07.801378 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:07.801722 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.802342 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.803565 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.804076 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.804219 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.804289 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:07.804525 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 11:10:07.804604 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260812 11:10:07.804960 30261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:07.801163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:54459 (local address 127.17.44.195:46469)
0812 11:10:07.801297 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:07.801301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:07.801481 (+ 180us) server_negotiation.cc:407] Connection header received
0812 11:10:07.801573 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:07.801576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:07.801630 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:07.801709 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:07.802007 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.802558 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.803836 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.804412 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.804524 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.804570 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:07.804775 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 11:10:07.804842 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 11:10:07.804986 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:07.801034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:46469 (local address 127.17.44.193:54459)
0812 11:10:07.801381 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:07.801394 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:07.801488 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:07.801730 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:07.801733 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:07.801751 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:07.801881 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:07.801889 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.802749 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.802752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:07.803706 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:07.803718 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.804526 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.804530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:07.804638 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.804667 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 11:10:07.804757 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 11:10:07.804827 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":1,"thread_start_us":94,"threads_started":1}
I20260812 11:10:07.805497 30282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:07.800500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:43485 (local address 127.17.44.193:55289)
0812 11:10:07.800956 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:07.800971 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:07.801123 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:07.801388 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:07.801390 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:07.801410 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:07.801576 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:07.801584 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.802471 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.802474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:07.803436 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:07.803447 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.804221 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.804224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:07.804324 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.804354 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 11:10:07.805319 (+ 965us) client_negotiation.cc:240] Negotiation successful
0812 11:10:07.805399 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":171,"threads_started":1}
I20260812 11:10:07.805639 30161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21896524823f412fbd1bd5249cc72a7a" candidate_uuid: "d7a6f415051b487eaefa063e0bb43960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" is_pre_election: true
I20260812 11:10:07.805992 30161 raft_consensus.cc:2454] T 21896524823f412fbd1bd5249cc72a7a P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7a6f415051b487eaefa063e0bb43960 in term 0.
I20260812 11:10:07.806303 29930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21896524823f412fbd1bd5249cc72a7a" candidate_uuid: "d7a6f415051b487eaefa063e0bb43960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03be7361f2d54061897ab2be44de9a2d" is_pre_election: true
I20260812 11:10:07.806607 29930 raft_consensus.cc:2454] T 21896524823f412fbd1bd5249cc72a7a P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7a6f415051b487eaefa063e0bb43960 in term 0.
I20260812 11:10:07.806622 29621 leader_election.cc:302] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82875c708dc345eba57b0ea8d0a4dc3f, d7a6f415051b487eaefa063e0bb43960; no voters:
I20260812 11:10:07.806962 30280 raft_consensus.cc:2781] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:10:07.807205 30280 raft_consensus.cc:482] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:10:07.807286 30280 raft_consensus.cc:3037] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:07.808478 30280 raft_consensus.cc:504] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } } peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } }
I20260812 11:10:07.808840 30280 leader_election.cc:288] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960 [CANDIDATE]: Term 1 election: Requested vote from peers 03be7361f2d54061897ab2be44de9a2d (127.17.44.194:43485), 82875c708dc345eba57b0ea8d0a4dc3f (127.17.44.195:46469)
I20260812 11:10:07.809311 29930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21896524823f412fbd1bd5249cc72a7a" candidate_uuid: "d7a6f415051b487eaefa063e0bb43960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03be7361f2d54061897ab2be44de9a2d"
I20260812 11:10:07.809454 29930 raft_consensus.cc:3037] T 21896524823f412fbd1bd5249cc72a7a P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:07.809454 30161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21896524823f412fbd1bd5249cc72a7a" candidate_uuid: "d7a6f415051b487eaefa063e0bb43960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82875c708dc345eba57b0ea8d0a4dc3f"
I20260812 11:10:07.809595 30161 raft_consensus.cc:3037] T 21896524823f412fbd1bd5249cc72a7a P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:07.810740 30161 raft_consensus.cc:2454] T 21896524823f412fbd1bd5249cc72a7a P 82875c708dc345eba57b0ea8d0a4dc3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7a6f415051b487eaefa063e0bb43960 in term 1.
I20260812 11:10:07.811084 29621 leader_election.cc:302] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82875c708dc345eba57b0ea8d0a4dc3f, d7a6f415051b487eaefa063e0bb43960; no voters:
I20260812 11:10:07.811090 29930 raft_consensus.cc:2454] T 21896524823f412fbd1bd5249cc72a7a P 03be7361f2d54061897ab2be44de9a2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7a6f415051b487eaefa063e0bb43960 in term 1.
I20260812 11:10:07.811249 30280 raft_consensus.cc:2781] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:10:07.811463 30280 raft_consensus.cc:686] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960 [term 1 LEADER]: Becoming Leader. State: Replica: d7a6f415051b487eaefa063e0bb43960, State: Running, Role: LEADER
I20260812 11:10:07.811846 30280 consensus_queue.cc:237] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } } peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } }
I20260812 11:10:07.815622 29484 catalog_manager.cc:5495] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7a6f415051b487eaefa063e0bb43960 (127.17.44.193). New cstate: current_term: 1 leader_uuid: "d7a6f415051b487eaefa063e0bb43960" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } health_report { overall_health: HEALTHY } } }
I20260812 11:10:07.833771 30263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:07.830064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55966 (local address 127.17.44.193:39365)
0812 11:10:07.830203 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:07.830211 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:07.830228 (+ 17us) server_negotiation.cc:407] Connection header received
0812 11:10:07.830310 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:07.830312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:07.830462 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:07.830606 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:10:07.830873 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.831441 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.832321 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:07.832538 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:07.832651 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:07.832945 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:10:07.833367 (+ 422us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:10:07.833495 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:07.833572 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 11:10:07.833627 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 11:10:07.839917 30161 raft_consensus.cc:1261] T 21896524823f412fbd1bd5249cc72a7a P 82875c708dc345eba57b0ea8d0a4dc3f [term 1 FOLLOWER]: Refusing update from remote peer d7a6f415051b487eaefa063e0bb43960: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:10:07.839955 29930 raft_consensus.cc:1261] T 21896524823f412fbd1bd5249cc72a7a P 03be7361f2d54061897ab2be44de9a2d [term 1 FOLLOWER]: Refusing update from remote peer d7a6f415051b487eaefa063e0bb43960: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:10:07.840692 30280 consensus_queue.cc:1035] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:10:07.840982 30284 consensus_queue.cc:1035] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:10:07.849196 30291 mvcc.cc:205] Tried to move back new op lower bound from 7317639200104845312 to 7317639200000364544. Current Snapshot: MvccSnapshot[applied={T|T < 7317639200104845312}]
I20260812 11:10:07.851573 30294 mvcc.cc:205] Tried to move back new op lower bound from 7317639200104845312 to 7317639200000364544. Current Snapshot: MvccSnapshot[applied={T|T < 7317639200104845312}]
I20260812 11:10:07.852279 30293 mvcc.cc:205] Tried to move back new op lower bound from 7317639200104845312 to 7317639200000364544. Current Snapshot: MvccSnapshot[applied={T|T < 7317639200104845312}]
I20260812 11:10:07.860313 29478 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:10:07.860611 29478 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:10:07.863788 29478 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c: Sending DeleteTablet for 3 replicas of tablet 21896524823f412fbd1bd5249cc72a7a
I20260812 11:10:07.865022 29915 tablet_service.cc:1514] Processing DeleteTablet for tablet 21896524823f412fbd1bd5249cc72a7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:10:07 UTC) from {username='oryx', principal='oryx/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:53518
I20260812 11:10:07.865100 29700 tablet_service.cc:1514] Processing DeleteTablet for tablet 21896524823f412fbd1bd5249cc72a7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:10:07 UTC) from {username='oryx', principal='oryx/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:55958
I20260812 11:10:07.865393 30124 tablet_service.cc:1514] Processing DeleteTablet for tablet 21896524823f412fbd1bd5249cc72a7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:10:07 UTC) from {username='oryx', principal='oryx/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:42292
I20260812 11:10:07.865648 29478 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 11:10:07.865963 30306 tablet_replica.cc:330] T 21896524823f412fbd1bd5249cc72a7a P 03be7361f2d54061897ab2be44de9a2d: stopping tablet replica
I20260812 11:10:07.866357 30307 tablet_replica.cc:330] T 21896524823f412fbd1bd5249cc72a7a P 82875c708dc345eba57b0ea8d0a4dc3f: stopping tablet replica
W20260812 11:10:07.866466 29478 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:10:07.866531 30306 raft_consensus.cc:2229] T 21896524823f412fbd1bd5249cc72a7a P 03be7361f2d54061897ab2be44de9a2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:10:07.866833 30307 raft_consensus.cc:2229] T 21896524823f412fbd1bd5249cc72a7a P 82875c708dc345eba57b0ea8d0a4dc3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:10:07.867225 30307 raft_consensus.cc:2258] T 21896524823f412fbd1bd5249cc72a7a P 82875c708dc345eba57b0ea8d0a4dc3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:10:07.867223 30306 raft_consensus.cc:2258] T 21896524823f412fbd1bd5249cc72a7a P 03be7361f2d54061897ab2be44de9a2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:10:07.868811 30307 ts_tablet_manager.cc:1905] T 21896524823f412fbd1bd5249cc72a7a P 82875c708dc345eba57b0ea8d0a4dc3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:10:07.868999 30306 ts_tablet_manager.cc:1905] T 21896524823f412fbd1bd5249cc72a7a P 03be7361f2d54061897ab2be44de9a2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:10:07.872274 30307 ts_tablet_manager.cc:1918] T 21896524823f412fbd1bd5249cc72a7a P 82875c708dc345eba57b0ea8d0a4dc3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:10:07.872277 30306 ts_tablet_manager.cc:1918] T 21896524823f412fbd1bd5249cc72a7a P 03be7361f2d54061897ab2be44de9a2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:10:07.872380 30306 log.cc:1196] T 21896524823f412fbd1bd5249cc72a7a P 03be7361f2d54061897ab2be44de9a2d: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/21896524823f412fbd1bd5249cc72a7a
I20260812 11:10:07.872978 30306 ts_tablet_manager.cc:1939] T 21896524823f412fbd1bd5249cc72a7a P 03be7361f2d54061897ab2be44de9a2d: Deleting consensus metadata
I20260812 11:10:07.872970 29915 tablet_service.cc:1467] Processing CreateTablet for tablet 2b9c581bfc754eecb3326ca421f23fd1 (DEFAULT_TABLE table=test-table [id=9628605677244d3ea41db2485da4dfd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:10:07.873011 29700 tablet_service.cc:1467] Processing CreateTablet for tablet 2b9c581bfc754eecb3326ca421f23fd1 (DEFAULT_TABLE table=test-table [id=9628605677244d3ea41db2485da4dfd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:10:07.873255 29915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b9c581bfc754eecb3326ca421f23fd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:07.873703 30305 tablet_replica.cc:330] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960: stopping tablet replica
I20260812 11:10:07.873884 29700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b9c581bfc754eecb3326ca421f23fd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:07.874115 30305 raft_consensus.cc:2229] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:10:07.874593 30305 raft_consensus.cc:2258] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:10:07.873430 30124 tablet_service.cc:1467] Processing CreateTablet for tablet 2b9c581bfc754eecb3326ca421f23fd1 (DEFAULT_TABLE table=test-table [id=9628605677244d3ea41db2485da4dfd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:10:07.874758 29466 catalog_manager.cc:4841] TS 03be7361f2d54061897ab2be44de9a2d (127.17.44.194:43485): tablet 21896524823f412fbd1bd5249cc72a7a (table test-table [id=4b7004b6aefc4677aea4b06fb26f0d5e]) successfully deleted
I20260812 11:10:07.874894 30124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b9c581bfc754eecb3326ca421f23fd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:07.877262 30270 tablet_bootstrap.cc:492] T 2b9c581bfc754eecb3326ca421f23fd1 P 03be7361f2d54061897ab2be44de9a2d: Bootstrap starting.
I20260812 11:10:07.877460 30264 tablet_bootstrap.cc:492] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f: Bootstrap starting.
I20260812 11:10:07.878259 30270 tablet_bootstrap.cc:654] T 2b9c581bfc754eecb3326ca421f23fd1 P 03be7361f2d54061897ab2be44de9a2d: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:07.878605 30264 tablet_bootstrap.cc:654] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:07.879556 30270 tablet_bootstrap.cc:492] T 2b9c581bfc754eecb3326ca421f23fd1 P 03be7361f2d54061897ab2be44de9a2d: No bootstrap required, opened a new log
I20260812 11:10:07.879647 30270 ts_tablet_manager.cc:1397] T 2b9c581bfc754eecb3326ca421f23fd1 P 03be7361f2d54061897ab2be44de9a2d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 11:10:07.880021 30264 tablet_bootstrap.cc:492] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f: No bootstrap required, opened a new log
I20260812 11:10:07.880134 30264 ts_tablet_manager.cc:1397] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 11:10:07.880076 30270 raft_consensus.cc:348] T 2b9c581bfc754eecb3326ca421f23fd1 P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:07.880205 30270 raft_consensus.cc:374] T 2b9c581bfc754eecb3326ca421f23fd1 P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:07.880247 30270 raft_consensus.cc:729] T 2b9c581bfc754eecb3326ca421f23fd1 P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03be7361f2d54061897ab2be44de9a2d, State: Initialized, Role: FOLLOWER
I20260812 11:10:07.880384 30270 consensus_queue.cc:260] T 2b9c581bfc754eecb3326ca421f23fd1 P 03be7361f2d54061897ab2be44de9a2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:07.880636 30270 ts_tablet_manager.cc:1428] T 2b9c581bfc754eecb3326ca421f23fd1 P 03be7361f2d54061897ab2be44de9a2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:10:07.880674 30264 raft_consensus.cc:348] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:07.880789 30264 raft_consensus.cc:374] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:07.880824 30264 raft_consensus.cc:729] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82875c708dc345eba57b0ea8d0a4dc3f, State: Initialized, Role: FOLLOWER
I20260812 11:10:07.880980 30264 consensus_queue.cc:260] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:07.881349 30264 ts_tablet_manager.cc:1428] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260812 11:10:07.881495 30307 log.cc:1196] T 21896524823f412fbd1bd5249cc72a7a P 82875c708dc345eba57b0ea8d0a4dc3f: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/21896524823f412fbd1bd5249cc72a7a
I20260812 11:10:07.881858 30307 ts_tablet_manager.cc:1939] T 21896524823f412fbd1bd5249cc72a7a P 82875c708dc345eba57b0ea8d0a4dc3f: Deleting consensus metadata
I20260812 11:10:07.876461 30267 tablet_bootstrap.cc:492] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960: Bootstrap starting.
I20260812 11:10:07.883457 30267 tablet_bootstrap.cc:654] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960: Neither blocks nor log segments found. Creating new log.
W20260812 11:10:07.884914 30011 tablet.cc:2369] T 2b9c581bfc754eecb3326ca421f23fd1 P 03be7361f2d54061897ab2be44de9a2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:10:07.885277 29466 catalog_manager.cc:4841] TS 82875c708dc345eba57b0ea8d0a4dc3f (127.17.44.195:46469): tablet 21896524823f412fbd1bd5249cc72a7a (table test-table [id=4b7004b6aefc4677aea4b06fb26f0d5e]) successfully deleted
I20260812 11:10:07.886370 30305 ts_tablet_manager.cc:1905] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:10:07.887745 30267 tablet_bootstrap.cc:492] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960: No bootstrap required, opened a new log
I20260812 11:10:07.887835 30267 ts_tablet_manager.cc:1397] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
I20260812 11:10:07.888273 30267 raft_consensus.cc:348] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:07.888381 30267 raft_consensus.cc:374] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:07.888411 30267 raft_consensus.cc:729] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7a6f415051b487eaefa063e0bb43960, State: Initialized, Role: FOLLOWER
I20260812 11:10:07.888522 30267 consensus_queue.cc:260] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:07.888767 30267 ts_tablet_manager.cc:1428] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 11:10:07.890740 30305 ts_tablet_manager.cc:1918] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:10:07.890836 30305 log.cc:1196] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-0/wal/wals/21896524823f412fbd1bd5249cc72a7a
I20260812 11:10:07.891180 30305 ts_tablet_manager.cc:1939] T 21896524823f412fbd1bd5249cc72a7a P d7a6f415051b487eaefa063e0bb43960: Deleting consensus metadata
I20260812 11:10:07.892299 29466 catalog_manager.cc:4841] TS d7a6f415051b487eaefa063e0bb43960 (127.17.44.193:39365): tablet 21896524823f412fbd1bd5249cc72a7a (table test-table [id=4b7004b6aefc4677aea4b06fb26f0d5e]) successfully deleted
W20260812 11:10:07.903352 30231 tablet.cc:2369] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:10:08.127637 30276 raft_consensus.cc:482] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:10:08.127674 30284 raft_consensus.cc:482] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:10:08.127846 30276 raft_consensus.cc:504] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.127890 30284 raft_consensus.cc:504] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.128525 30284 leader_election.cc:288] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03be7361f2d54061897ab2be44de9a2d (127.17.44.194:43485), 82875c708dc345eba57b0ea8d0a4dc3f (127.17.44.195:46469)
I20260812 11:10:08.129192 30161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b9c581bfc754eecb3326ca421f23fd1" candidate_uuid: "d7a6f415051b487eaefa063e0bb43960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" is_pre_election: true
I20260812 11:10:08.129236 29930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b9c581bfc754eecb3326ca421f23fd1" candidate_uuid: "d7a6f415051b487eaefa063e0bb43960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03be7361f2d54061897ab2be44de9a2d" is_pre_election: true
I20260812 11:10:08.129344 30276 leader_election.cc:288] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7a6f415051b487eaefa063e0bb43960 (127.17.44.193:39365), 03be7361f2d54061897ab2be44de9a2d (127.17.44.194:43485)
I20260812 11:10:08.129354 30161 raft_consensus.cc:2454] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7a6f415051b487eaefa063e0bb43960 in term 0.
I20260812 11:10:08.129463 29930 raft_consensus.cc:2454] T 2b9c581bfc754eecb3326ca421f23fd1 P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7a6f415051b487eaefa063e0bb43960 in term 0.
I20260812 11:10:08.130046 29621 leader_election.cc:302] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03be7361f2d54061897ab2be44de9a2d, d7a6f415051b487eaefa063e0bb43960; no voters:
I20260812 11:10:08.130422 30284 raft_consensus.cc:2781] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:10:08.130501 30284 raft_consensus.cc:482] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:10:08.130548 30284 raft_consensus.cc:3037] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:08.132011 30284 raft_consensus.cc:504] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.132441 30284 leader_election.cc:288] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960 [CANDIDATE]: Term 1 election: Requested vote from peers 03be7361f2d54061897ab2be44de9a2d (127.17.44.194:43485), 82875c708dc345eba57b0ea8d0a4dc3f (127.17.44.195:46469)
I20260812 11:10:08.132983 29930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b9c581bfc754eecb3326ca421f23fd1" candidate_uuid: "d7a6f415051b487eaefa063e0bb43960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03be7361f2d54061897ab2be44de9a2d"
I20260812 11:10:08.133142 29930 raft_consensus.cc:3037] T 2b9c581bfc754eecb3326ca421f23fd1 P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:08.134295 29930 raft_consensus.cc:2454] T 2b9c581bfc754eecb3326ca421f23fd1 P 03be7361f2d54061897ab2be44de9a2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7a6f415051b487eaefa063e0bb43960 in term 1.
I20260812 11:10:08.134660 30069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:08.129644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:39365 (local address 127.17.44.195:55873)
0812 11:10:08.129832 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:08.129850 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:08.129942 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:08.130546 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:08.130550 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:08.130574 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:08.130717 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:08.130727 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.131818 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.131823 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:08.133193 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:08.133206 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.134190 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.134194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:08.134306 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:08.134335 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 11:10:08.134418 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 11:10:08.134497 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":9}
I20260812 11:10:08.134805 30161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b9c581bfc754eecb3326ca421f23fd1" candidate_uuid: "d7a6f415051b487eaefa063e0bb43960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82875c708dc345eba57b0ea8d0a4dc3f"
I20260812 11:10:08.134847 30258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:08.130313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:59413 (local address 127.17.44.194:43485)
0812 11:10:08.130465 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:08.130469 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:08.130483 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:10:08.130646 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:08.130649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:08.130699 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:08.130789 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:08.131151 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.131686 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.133531 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.134115 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.134195 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:08.134237 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:08.134637 (+ 400us) server_negotiation.cc:299] Negotiation successful
0812 11:10:08.134727 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 11:10:08.134917 30161 raft_consensus.cc:3037] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:08.135167 30263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:08.129903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:55873 (local address 127.17.44.193:39365)
0812 11:10:08.130048 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:08.130053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:08.130068 (+ 15us) server_negotiation.cc:407] Connection header received
0812 11:10:08.130117 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:08.130120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:08.130173 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:08.130387 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:08.130874 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.131685 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.133338 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.133959 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.134070 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:08.134127 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:08.134962 (+ 835us) server_negotiation.cc:299] Negotiation successful
0812 11:10:08.135067 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260812 11:10:08.135383 29803 tablet.cc:2369] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:10:08.135663 29727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b9c581bfc754eecb3326ca421f23fd1" candidate_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7a6f415051b487eaefa063e0bb43960" is_pre_election: true
I20260812 11:10:08.135896 29727 raft_consensus.cc:2379] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 82875c708dc345eba57b0ea8d0a4dc3f in current term 1: Already voted for candidate d7a6f415051b487eaefa063e0bb43960 in this term.
I20260812 11:10:08.136089 30161 raft_consensus.cc:2454] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7a6f415051b487eaefa063e0bb43960 in term 1.
I20260812 11:10:08.136339 30320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:08.129903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:43485 (local address 127.17.44.195:59413)
0812 11:10:08.130366 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:08.130381 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:08.130487 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:08.130805 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:08.130808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:08.130825 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:08.131017 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:08.131024 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.131831 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.131833 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:08.133409 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:08.133417 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.134336 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.134339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:08.134502 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:08.134523 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 11:10:08.136063 (+ 1540us) client_negotiation.cc:240] Negotiation successful
0812 11:10:08.136144 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":161,"thread_start_us":97,"threads_started":1}
I20260812 11:10:08.136413 29621 leader_election.cc:302] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82875c708dc345eba57b0ea8d0a4dc3f, d7a6f415051b487eaefa063e0bb43960; no voters:
I20260812 11:10:08.136682 30284 raft_consensus.cc:2781] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:10:08.136914 30284 raft_consensus.cc:686] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960 [term 1 LEADER]: Becoming Leader. State: Replica: d7a6f415051b487eaefa063e0bb43960, State: Running, Role: LEADER
I20260812 11:10:08.137117 30284 consensus_queue.cc:237] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.137243 29930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b9c581bfc754eecb3326ca421f23fd1" candidate_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03be7361f2d54061897ab2be44de9a2d" is_pre_election: true
I20260812 11:10:08.137434 29930 raft_consensus.cc:2379] T 2b9c581bfc754eecb3326ca421f23fd1 P 03be7361f2d54061897ab2be44de9a2d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 82875c708dc345eba57b0ea8d0a4dc3f in current term 1: Already voted for candidate d7a6f415051b487eaefa063e0bb43960 in this term.
I20260812 11:10:08.137823 30059 leader_election.cc:302] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 82875c708dc345eba57b0ea8d0a4dc3f; no voters: 03be7361f2d54061897ab2be44de9a2d, d7a6f415051b487eaefa063e0bb43960
I20260812 11:10:08.138125 30276 raft_consensus.cc:2726] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 11:10:08.139679 29478 catalog_manager.cc:5495] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7a6f415051b487eaefa063e0bb43960 (127.17.44.193). New cstate: current_term: 1 leader_uuid: "d7a6f415051b487eaefa063e0bb43960" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } health_report { overall_health: UNKNOWN } } }
Aug 12 11:10:08 dist-test-slave-drl0 krb5kdc[29413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533006, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.254@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM
I20260812 11:10:08.202032 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:08.192882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39408 (local address 127.17.44.254:38053)
0812 11:10:08.193003 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:08.193006 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:08.193038 (+ 32us) server_negotiation.cc:407] Connection header received
0812 11:10:08.193230 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:08.193233 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:08.193278 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:08.193363 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:08.194077 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.194609 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.195568 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.195818 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.195911 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:08.198407 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:08.198410 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:08.198412 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:08.198432 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:08.199949 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:08.200474 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:08.200481 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:08.200483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:08.200566 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:08.200877 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:08.200883 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:08.200885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:08.201080 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:08.201197 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:08.201674 (+ 477us) server_negotiation.cc:299] Negotiation successful
0812 11:10:08.201890 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":83,"server-negotiator.queue_time_us":49}
I20260812 11:10:08.202032 30327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:08.192657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:38053 (local address 127.0.0.1:39408)
0812 11:10:08.192973 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:08.192987 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:08.193087 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:08.193426 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:08.193429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:08.193835 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:08.193939 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:08.193945 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.194789 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.194792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:08.195405 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:08.195414 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.195960 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.195963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:08.196026 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:08.196721 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:08.196734 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:08.198252 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:08.200077 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:08.200090 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:08.200097 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:08.200360 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:08.200678 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:08.200681 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:08.200683 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:08.200777 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:08.201209 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:08.201217 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:08.201413 (+ 196us) client_negotiation.cc:769] Sending connection context
0812 11:10:08.201614 (+ 201us) client_negotiation.cc:240] Negotiation successful
0812 11:10:08.201819 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":120,"threads_started":1}
I20260812 11:10:08.206975 29478 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:39408:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 11:10:08.207759 29478 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:10:08.213481 29915 tablet_service.cc:1467] Processing CreateTablet for tablet 99670244e924427296cacf5944250932 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=981285b76fab422baf433c30302abb7a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 11:10:08.213481 30124 tablet_service.cc:1467] Processing CreateTablet for tablet 99670244e924427296cacf5944250932 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=981285b76fab422baf433c30302abb7a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 11:10:08.213459 29700 tablet_service.cc:1467] Processing CreateTablet for tablet 99670244e924427296cacf5944250932 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=981285b76fab422baf433c30302abb7a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 11:10:08.213869 29700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99670244e924427296cacf5944250932. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:08.213874 29915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99670244e924427296cacf5944250932. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:08.213874 30124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99670244e924427296cacf5944250932. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:08.216724 30267 tablet_bootstrap.cc:492] T 99670244e924427296cacf5944250932 P d7a6f415051b487eaefa063e0bb43960: Bootstrap starting.
I20260812 11:10:08.216893 30264 tablet_bootstrap.cc:492] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f: Bootstrap starting.
I20260812 11:10:08.216907 30270 tablet_bootstrap.cc:492] T 99670244e924427296cacf5944250932 P 03be7361f2d54061897ab2be44de9a2d: Bootstrap starting.
I20260812 11:10:08.217516 30267 tablet_bootstrap.cc:654] T 99670244e924427296cacf5944250932 P d7a6f415051b487eaefa063e0bb43960: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:08.217824 30264 tablet_bootstrap.cc:654] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:08.217824 30270 tablet_bootstrap.cc:654] T 99670244e924427296cacf5944250932 P 03be7361f2d54061897ab2be44de9a2d: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:08.219244 30264 tablet_bootstrap.cc:492] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f: No bootstrap required, opened a new log
I20260812 11:10:08.219455 30264 ts_tablet_manager.cc:1397] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260812 11:10:08.219630 30267 tablet_bootstrap.cc:492] T 99670244e924427296cacf5944250932 P d7a6f415051b487eaefa063e0bb43960: No bootstrap required, opened a new log
I20260812 11:10:08.219729 30267 ts_tablet_manager.cc:1397] T 99670244e924427296cacf5944250932 P d7a6f415051b487eaefa063e0bb43960: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 11:10:08.220240 30267 raft_consensus.cc:348] T 99670244e924427296cacf5944250932 P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.220388 30267 raft_consensus.cc:374] T 99670244e924427296cacf5944250932 P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:08.220319 30264 raft_consensus.cc:348] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.220443 30267 raft_consensus.cc:729] T 99670244e924427296cacf5944250932 P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7a6f415051b487eaefa063e0bb43960, State: Initialized, Role: FOLLOWER
I20260812 11:10:08.220456 30264 raft_consensus.cc:374] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:08.220495 30264 raft_consensus.cc:729] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82875c708dc345eba57b0ea8d0a4dc3f, State: Initialized, Role: FOLLOWER
I20260812 11:10:08.220634 30264 consensus_queue.cc:260] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.220990 30264 ts_tablet_manager.cc:1428] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260812 11:10:08.220922 30276 tablet_replica.cc:439] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } } }
I20260812 11:10:08.221086 30276 tablet_replica.cc:442] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:10:08.220671 30267 consensus_queue.cc:260] T 99670244e924427296cacf5944250932 P d7a6f415051b487eaefa063e0bb43960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.221354 30267 ts_tablet_manager.cc:1428] T 99670244e924427296cacf5944250932 P d7a6f415051b487eaefa063e0bb43960: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260812 11:10:08.221606 30284 tablet_replica.cc:439] T 99670244e924427296cacf5944250932 P d7a6f415051b487eaefa063e0bb43960: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } } }
I20260812 11:10:08.221752 30284 tablet_replica.cc:442] T 99670244e924427296cacf5944250932 P d7a6f415051b487eaefa063e0bb43960: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:10:08.221843 30270 tablet_bootstrap.cc:492] T 99670244e924427296cacf5944250932 P 03be7361f2d54061897ab2be44de9a2d: No bootstrap required, opened a new log
I20260812 11:10:08.221939 30270 ts_tablet_manager.cc:1397] T 99670244e924427296cacf5944250932 P 03be7361f2d54061897ab2be44de9a2d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 11:10:08.222486 30270 raft_consensus.cc:348] T 99670244e924427296cacf5944250932 P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.222641 30270 raft_consensus.cc:374] T 99670244e924427296cacf5944250932 P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:08.222678 30270 raft_consensus.cc:729] T 99670244e924427296cacf5944250932 P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03be7361f2d54061897ab2be44de9a2d, State: Initialized, Role: FOLLOWER
I20260812 11:10:08.222844 30270 consensus_queue.cc:260] T 99670244e924427296cacf5944250932 P 03be7361f2d54061897ab2be44de9a2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.223176 30270 ts_tablet_manager.cc:1428] T 99670244e924427296cacf5944250932 P 03be7361f2d54061897ab2be44de9a2d: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260812 11:10:08.223440 30279 tablet_replica.cc:439] T 99670244e924427296cacf5944250932 P 03be7361f2d54061897ab2be44de9a2d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } } }
I20260812 11:10:08.223580 30279 tablet_replica.cc:442] T 99670244e924427296cacf5944250932 P 03be7361f2d54061897ab2be44de9a2d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:10:08.249544 30276 raft_consensus.cc:482] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:10:08.249681 30276 raft_consensus.cc:504] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.250165 30276 leader_election.cc:288] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7a6f415051b487eaefa063e0bb43960 (127.17.44.193:39365), 03be7361f2d54061897ab2be44de9a2d (127.17.44.194:43485)
I20260812 11:10:08.250741 29727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99670244e924427296cacf5944250932" candidate_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7a6f415051b487eaefa063e0bb43960" is_pre_election: true
I20260812 11:10:08.250749 29930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99670244e924427296cacf5944250932" candidate_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03be7361f2d54061897ab2be44de9a2d" is_pre_election: true
I20260812 11:10:08.250944 29930 raft_consensus.cc:2454] T 99670244e924427296cacf5944250932 P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82875c708dc345eba57b0ea8d0a4dc3f in term 0.
I20260812 11:10:08.250986 29727 raft_consensus.cc:2454] T 99670244e924427296cacf5944250932 P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82875c708dc345eba57b0ea8d0a4dc3f in term 0.
I20260812 11:10:08.251288 30059 leader_election.cc:302] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03be7361f2d54061897ab2be44de9a2d, 82875c708dc345eba57b0ea8d0a4dc3f; no voters:
I20260812 11:10:08.251461 30276 raft_consensus.cc:2781] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:10:08.251538 30276 raft_consensus.cc:482] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:10:08.251600 30276 raft_consensus.cc:3037] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:08.252810 30276 raft_consensus.cc:504] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.253170 30276 leader_election.cc:288] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f [CANDIDATE]: Term 1 election: Requested vote from peers d7a6f415051b487eaefa063e0bb43960 (127.17.44.193:39365), 03be7361f2d54061897ab2be44de9a2d (127.17.44.194:43485)
I20260812 11:10:08.253532 29930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99670244e924427296cacf5944250932" candidate_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03be7361f2d54061897ab2be44de9a2d"
I20260812 11:10:08.253535 29727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99670244e924427296cacf5944250932" candidate_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7a6f415051b487eaefa063e0bb43960"
I20260812 11:10:08.253667 29930 raft_consensus.cc:3037] T 99670244e924427296cacf5944250932 P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:08.253684 29727 raft_consensus.cc:3037] T 99670244e924427296cacf5944250932 P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:08.254539 29930 raft_consensus.cc:2454] T 99670244e924427296cacf5944250932 P 03be7361f2d54061897ab2be44de9a2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82875c708dc345eba57b0ea8d0a4dc3f in term 1.
I20260812 11:10:08.254778 29727 raft_consensus.cc:2454] T 99670244e924427296cacf5944250932 P d7a6f415051b487eaefa063e0bb43960 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82875c708dc345eba57b0ea8d0a4dc3f in term 1.
I20260812 11:10:08.254866 30059 leader_election.cc:302] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03be7361f2d54061897ab2be44de9a2d, 82875c708dc345eba57b0ea8d0a4dc3f; no voters:
I20260812 11:10:08.255074 30276 raft_consensus.cc:2781] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:10:08.255270 30276 raft_consensus.cc:686] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f [term 1 LEADER]: Becoming Leader. State: Replica: 82875c708dc345eba57b0ea8d0a4dc3f, State: Running, Role: LEADER
I20260812 11:10:08.255548 30276 consensus_queue.cc:237] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.256714 30331 tablet_replica.cc:439] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f: TxnStatusTablet state changed. Reason: New leader 82875c708dc345eba57b0ea8d0a4dc3f. Latest consensus state: current_term: 1 leader_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } } }
I20260812 11:10:08.256847 30331 tablet_replica.cc:442] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f: This TxnStatusTablet replica's current role is: LEADER
I20260812 11:10:08.257366 30333 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 11:10:08.257540 30333 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 11:10:08.257982 29479 catalog_manager.cc:5495] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f reported cstate change: term changed from 0 to 1, leader changed from <none> to 82875c708dc345eba57b0ea8d0a4dc3f (127.17.44.195). New cstate: current_term: 1 leader_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } health_report { overall_health: HEALTHY } } }
I20260812 11:10:08.273944 30261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:08.269491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42296 (local address 127.17.44.195:46469)
0812 11:10:08.269661 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:08.269664 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:08.269678 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:10:08.269936 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:08.269940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:08.270037 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:08.270184 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:10:08.270557 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.271137 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.272426 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.272647 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.272786 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:08.273063 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:10:08.273530 (+ 467us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:10:08.273633 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:08.273779 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 11:10:08.273840 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 11:10:08.274006 30327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:08.269158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:46469 (local address 127.0.0.1:42296)
0812 11:10:08.269412 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:08.269428 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:08.269533 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:08.270234 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:08.270238 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:08.270247 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 11:10:08.270373 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:08.270380 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.271251 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.271254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:08.272233 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:08.272245 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.272767 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.272771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:08.272864 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:08.272925 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 11:10:08.273631 (+ 706us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 11:10:08.273646 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 11:10:08.273731 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 11:10:08.273828 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
I20260812 11:10:08.277312 29930 raft_consensus.cc:1261] T 99670244e924427296cacf5944250932 P 03be7361f2d54061897ab2be44de9a2d [term 1 FOLLOWER]: Refusing update from remote peer 82875c708dc345eba57b0ea8d0a4dc3f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:10:08.277462 29727 raft_consensus.cc:1261] T 99670244e924427296cacf5944250932 P d7a6f415051b487eaefa063e0bb43960 [term 1 FOLLOWER]: Refusing update from remote peer 82875c708dc345eba57b0ea8d0a4dc3f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:10:08.277923 30331 consensus_queue.cc:1035] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:10:08.278085 30276 consensus_queue.cc:1035] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:10:08.280156 30279 tablet_replica.cc:439] T 99670244e924427296cacf5944250932 P 03be7361f2d54061897ab2be44de9a2d: TxnStatusTablet state changed. Reason: New leader 82875c708dc345eba57b0ea8d0a4dc3f. Latest consensus state: current_term: 1 leader_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } } }
I20260812 11:10:08.280293 30279 tablet_replica.cc:442] T 99670244e924427296cacf5944250932 P 03be7361f2d54061897ab2be44de9a2d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:10:08.280285 30284 tablet_replica.cc:439] T 99670244e924427296cacf5944250932 P d7a6f415051b487eaefa063e0bb43960: TxnStatusTablet state changed. Reason: New leader 82875c708dc345eba57b0ea8d0a4dc3f. Latest consensus state: current_term: 1 leader_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } } }
I20260812 11:10:08.280403 30284 tablet_replica.cc:442] T 99670244e924427296cacf5944250932 P d7a6f415051b487eaefa063e0bb43960: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:10:08.283226 30276 tablet_replica.cc:439] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } } }
I20260812 11:10:08.283349 30276 tablet_replica.cc:442] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f: This TxnStatusTablet replica's current role is: LEADER
I20260812 11:10:08.284672 30331 tablet_replica.cc:439] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } } }
I20260812 11:10:08.284781 30331 tablet_replica.cc:442] T 99670244e924427296cacf5944250932 P 82875c708dc345eba57b0ea8d0a4dc3f: This TxnStatusTablet replica's current role is: LEADER
I20260812 11:10:08.285288 30284 tablet_replica.cc:439] T 99670244e924427296cacf5944250932 P d7a6f415051b487eaefa063e0bb43960: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } } }
I20260812 11:10:08.285405 30284 tablet_replica.cc:442] T 99670244e924427296cacf5944250932 P d7a6f415051b487eaefa063e0bb43960: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:10:08.286103 30279 tablet_replica.cc:439] T 99670244e924427296cacf5944250932 P 03be7361f2d54061897ab2be44de9a2d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } } }
I20260812 11:10:08.286231 30279 tablet_replica.cc:442] T 99670244e924427296cacf5944250932 P 03be7361f2d54061897ab2be44de9a2d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 11:10:08.301546 30261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:08.297704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:60067 (local address 127.17.44.195:46469)
0812 11:10:08.297833 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:08.297841 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:08.298102 (+ 261us) server_negotiation.cc:407] Connection header received
0812 11:10:08.298261 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:08.298264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:08.298319 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:08.298434 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:10:08.298702 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.299337 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.300403 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.300668 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.300800 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:08.301109 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:10:08.301185 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:10:08.301269 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:08.301362 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 11:10:08.301410 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 11:10:08.301569 30341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:08.297583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:46469 (local address 127.17.44.193:60067)
0812 11:10:08.298015 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:08.298026 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:08.298150 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:08.298462 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:08.298465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:08.298474 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 11:10:08.298591 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:08.298598 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.299461 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.299466 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:08.300179 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:08.300189 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.300800 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.300804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:08.300896 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:08.300964 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 11:10:08.301276 (+ 312us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 11:10:08.301286 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 11:10:08.301353 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 11:10:08.301435 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":134,"threads_started":1}
I20260812 11:10:08.308131 29930 raft_consensus.cc:1261] T 2b9c581bfc754eecb3326ca421f23fd1 P 03be7361f2d54061897ab2be44de9a2d [term 1 FOLLOWER]: Refusing update from remote peer d7a6f415051b487eaefa063e0bb43960: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:10:08.308131 30161 raft_consensus.cc:1261] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f [term 1 FOLLOWER]: Refusing update from remote peer d7a6f415051b487eaefa063e0bb43960: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:10:08.308568 30280 consensus_queue.cc:1035] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:10:08.308696 30284 consensus_queue.cc:1035] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:10:08.334121 30263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:08.329665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:41105 (local address 127.17.44.193:39365)
0812 11:10:08.329790 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:08.329794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:08.330025 (+ 231us) server_negotiation.cc:407] Connection header received
0812 11:10:08.330227 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:08.330229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:08.330319 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:08.330410 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:10:08.330697 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.331420 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.332486 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.332893 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.333011 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:08.333329 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:10:08.333418 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:10:08.333502 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:08.333911 (+ 409us) server_negotiation.cc:299] Negotiation successful
0812 11:10:08.333979 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 11:10:08.334640 30352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:08.329426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:39365 (local address 127.17.44.195:41105)
0812 11:10:08.329838 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:08.329854 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:08.330073 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:08.330428 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:08.330431 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:08.330438 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 11:10:08.330573 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:08.330579 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.331570 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.331573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:08.332361 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:08.332371 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.333061 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.333065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:08.333143 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:08.333190 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 11:10:08.333764 (+ 574us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 11:10:08.333772 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 11:10:08.334361 (+ 589us) client_negotiation.cc:240] Negotiation successful
0812 11:10:08.334456 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":112,"threads_started":1}
I20260812 11:10:08.366719 29478 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:10:08.366942 29478 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:10:08.369433 29478 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c: Sending DeleteTablet for 3 replicas of tablet 2b9c581bfc754eecb3326ca421f23fd1
I20260812 11:10:08.370410 29915 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b9c581bfc754eecb3326ca421f23fd1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:10:08 UTC) from {username='oryx', principal='oryx/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:53518
I20260812 11:10:08.370724 30306 tablet_replica.cc:330] T 2b9c581bfc754eecb3326ca421f23fd1 P 03be7361f2d54061897ab2be44de9a2d: stopping tablet replica
I20260812 11:10:08.370853 30306 raft_consensus.cc:2229] T 2b9c581bfc754eecb3326ca421f23fd1 P 03be7361f2d54061897ab2be44de9a2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:10:08.371356 30306 raft_consensus.cc:2258] T 2b9c581bfc754eecb3326ca421f23fd1 P 03be7361f2d54061897ab2be44de9a2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:10:08.372325 30306 ts_tablet_manager.cc:1905] T 2b9c581bfc754eecb3326ca421f23fd1 P 03be7361f2d54061897ab2be44de9a2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:10:08.375530 30306 ts_tablet_manager.cc:1918] T 2b9c581bfc754eecb3326ca421f23fd1 P 03be7361f2d54061897ab2be44de9a2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:10:08.375604 30306 log.cc:1196] T 2b9c581bfc754eecb3326ca421f23fd1 P 03be7361f2d54061897ab2be44de9a2d: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/2b9c581bfc754eecb3326ca421f23fd1
I20260812 11:10:08.375886 30306 ts_tablet_manager.cc:1939] T 2b9c581bfc754eecb3326ca421f23fd1 P 03be7361f2d54061897ab2be44de9a2d: Deleting consensus metadata
I20260812 11:10:08.376662 29466 catalog_manager.cc:4841] TS 03be7361f2d54061897ab2be44de9a2d (127.17.44.194:43485): tablet 2b9c581bfc754eecb3326ca421f23fd1 (table test-table [id=9628605677244d3ea41db2485da4dfd5]) successfully deleted
I20260812 11:10:08.376879 30124 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b9c581bfc754eecb3326ca421f23fd1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:10:08 UTC) from {username='oryx', principal='oryx/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:42292
I20260812 11:10:08.376936 29700 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b9c581bfc754eecb3326ca421f23fd1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:10:08 UTC) from {username='oryx', principal='oryx/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:55958
I20260812 11:10:08.377079 30307 tablet_replica.cc:330] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f: stopping tablet replica
I20260812 11:10:08.377141 30305 tablet_replica.cc:330] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960: stopping tablet replica
I20260812 11:10:08.377206 30307 raft_consensus.cc:2229] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:10:08.377262 30305 raft_consensus.cc:2229] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:10:08.377341 30307 raft_consensus.cc:2258] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:10:08.377480 30305 raft_consensus.cc:2258] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:10:08.378189 30305 ts_tablet_manager.cc:1905] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:10:08.378189 30307 ts_tablet_manager.cc:1905] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:10:08.380983 30307 ts_tablet_manager.cc:1918] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:10:08.380983 30305 ts_tablet_manager.cc:1918] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:10:08.381059 30305 log.cc:1196] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-0/wal/wals/2b9c581bfc754eecb3326ca421f23fd1
I20260812 11:10:08.381059 30307 log.cc:1196] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/2b9c581bfc754eecb3326ca421f23fd1
I20260812 11:10:08.381343 30305 ts_tablet_manager.cc:1939] T 2b9c581bfc754eecb3326ca421f23fd1 P d7a6f415051b487eaefa063e0bb43960: Deleting consensus metadata
I20260812 11:10:08.381342 30307 ts_tablet_manager.cc:1939] T 2b9c581bfc754eecb3326ca421f23fd1 P 82875c708dc345eba57b0ea8d0a4dc3f: Deleting consensus metadata
I20260812 11:10:08.382100 29466 catalog_manager.cc:4841] TS d7a6f415051b487eaefa063e0bb43960 (127.17.44.193:39365): tablet 2b9c581bfc754eecb3326ca421f23fd1 (table test-table [id=9628605677244d3ea41db2485da4dfd5]) successfully deleted
I20260812 11:10:08.382257 29466 catalog_manager.cc:4841] TS 82875c708dc345eba57b0ea8d0a4dc3f (127.17.44.195:46469): tablet 2b9c581bfc754eecb3326ca421f23fd1 (table test-table [id=9628605677244d3ea41db2485da4dfd5]) successfully deleted
Aug 12 11:10:08 dist-test-slave-drl0 krb5kdc[29413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 11:10:08 dist-test-slave-drl0 krb5kdc[29413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:08 dist-test-slave-drl0 krb5kdc[29413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 11:10:08.409032 29559 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:08.400611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39420 (local address 127.17.44.254:38053)
0812 11:10:08.400753 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:08.400759 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:08.401005 (+ 246us) server_negotiation.cc:407] Connection header received
0812 11:10:08.401157 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:08.401163 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:08.401241 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:08.401452 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:08.402287 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.403189 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.404408 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.404755 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.404824 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:08.408899 (+ 4075us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39420: BlockingRecv error: recv got EOF from 127.0.0.1:39420 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 11:10:08 dist-test-slave-drl0 krb5kdc[29413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM
I20260812 11:10:08.421289 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:08.411026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39424 (local address 127.17.44.254:38053)
0812 11:10:08.411161 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:08.411165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:08.411349 (+ 184us) server_negotiation.cc:407] Connection header received
0812 11:10:08.411459 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:08.411461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:08.411499 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:08.411565 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:08.412336 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.412828 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.413895 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.414129 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.414214 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:08.416871 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:08.416876 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:08.416880 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:08.416915 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:08.418683 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:08.419225 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:08.419229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:08.419231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:08.419323 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:08.419607 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:08.419610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:08.419612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:08.419840 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:08.419989 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:08.420488 (+ 499us) server_negotiation.cc:299] Negotiation successful
0812 11:10:08.421125 (+ 637us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 11:10:08.424966 29478 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39424:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:10:08.425482 29478 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:10:08.431200 29700 tablet_service.cc:1467] Processing CreateTablet for tablet 9153228cffb94691a13a7e5698a5b12e (DEFAULT_TABLE table=test-table [id=4f31833bb73547fba2c17ea3da335032]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:10:08.431761 29700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9153228cffb94691a13a7e5698a5b12e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:08.434667 30267 tablet_bootstrap.cc:492] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960: Bootstrap starting.
I20260812 11:10:08.435077 29915 tablet_service.cc:1467] Processing CreateTablet for tablet 9153228cffb94691a13a7e5698a5b12e (DEFAULT_TABLE table=test-table [id=4f31833bb73547fba2c17ea3da335032]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:10:08.435185 30124 tablet_service.cc:1467] Processing CreateTablet for tablet 9153228cffb94691a13a7e5698a5b12e (DEFAULT_TABLE table=test-table [id=4f31833bb73547fba2c17ea3da335032]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:10:08.435405 29915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9153228cffb94691a13a7e5698a5b12e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:08.435465 30124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9153228cffb94691a13a7e5698a5b12e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:08.435657 30267 tablet_bootstrap.cc:654] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:08.438063 30270 tablet_bootstrap.cc:492] T 9153228cffb94691a13a7e5698a5b12e P 03be7361f2d54061897ab2be44de9a2d: Bootstrap starting.
I20260812 11:10:08.438352 30267 tablet_bootstrap.cc:492] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960: No bootstrap required, opened a new log
I20260812 11:10:08.438450 30267 ts_tablet_manager.cc:1397] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 11:10:08.438961 30267 raft_consensus.cc:348] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.439096 30267 raft_consensus.cc:374] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:08.439108 30264 tablet_bootstrap.cc:492] T 9153228cffb94691a13a7e5698a5b12e P 82875c708dc345eba57b0ea8d0a4dc3f: Bootstrap starting.
I20260812 11:10:08.439137 30267 raft_consensus.cc:729] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7a6f415051b487eaefa063e0bb43960, State: Initialized, Role: FOLLOWER
I20260812 11:10:08.439263 30267 consensus_queue.cc:260] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.439558 30267 ts_tablet_manager.cc:1428] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:10:08.440243 30264 tablet_bootstrap.cc:654] T 9153228cffb94691a13a7e5698a5b12e P 82875c708dc345eba57b0ea8d0a4dc3f: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:08.440735 30270 tablet_bootstrap.cc:654] T 9153228cffb94691a13a7e5698a5b12e P 03be7361f2d54061897ab2be44de9a2d: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:08.441605 30264 tablet_bootstrap.cc:492] T 9153228cffb94691a13a7e5698a5b12e P 82875c708dc345eba57b0ea8d0a4dc3f: No bootstrap required, opened a new log
I20260812 11:10:08.441699 30264 ts_tablet_manager.cc:1397] T 9153228cffb94691a13a7e5698a5b12e P 82875c708dc345eba57b0ea8d0a4dc3f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 11:10:08.442376 30264 raft_consensus.cc:348] T 9153228cffb94691a13a7e5698a5b12e P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.442662 30270 tablet_bootstrap.cc:492] T 9153228cffb94691a13a7e5698a5b12e P 03be7361f2d54061897ab2be44de9a2d: No bootstrap required, opened a new log
I20260812 11:10:08.442696 30264 raft_consensus.cc:374] T 9153228cffb94691a13a7e5698a5b12e P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:08.442754 30270 ts_tablet_manager.cc:1397] T 9153228cffb94691a13a7e5698a5b12e P 03be7361f2d54061897ab2be44de9a2d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260812 11:10:08.442759 30264 raft_consensus.cc:729] T 9153228cffb94691a13a7e5698a5b12e P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82875c708dc345eba57b0ea8d0a4dc3f, State: Initialized, Role: FOLLOWER
I20260812 11:10:08.442898 30264 consensus_queue.cc:260] T 9153228cffb94691a13a7e5698a5b12e P 82875c708dc345eba57b0ea8d0a4dc3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.443233 30264 ts_tablet_manager.cc:1428] T 9153228cffb94691a13a7e5698a5b12e P 82875c708dc345eba57b0ea8d0a4dc3f: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 11:10:08.443288 30270 raft_consensus.cc:348] T 9153228cffb94691a13a7e5698a5b12e P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.443406 30270 raft_consensus.cc:374] T 9153228cffb94691a13a7e5698a5b12e P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:08.443449 30270 raft_consensus.cc:729] T 9153228cffb94691a13a7e5698a5b12e P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03be7361f2d54061897ab2be44de9a2d, State: Initialized, Role: FOLLOWER
I20260812 11:10:08.443571 30270 consensus_queue.cc:260] T 9153228cffb94691a13a7e5698a5b12e P 03be7361f2d54061897ab2be44de9a2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.443849 30270 ts_tablet_manager.cc:1428] T 9153228cffb94691a13a7e5698a5b12e P 03be7361f2d54061897ab2be44de9a2d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 11:10:08.481405 30284 raft_consensus.cc:482] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:10:08.481732 30284 raft_consensus.cc:504] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.482190 30284 leader_election.cc:288] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03be7361f2d54061897ab2be44de9a2d (127.17.44.194:43485), 82875c708dc345eba57b0ea8d0a4dc3f (127.17.44.195:46469)
I20260812 11:10:08.482911 30161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9153228cffb94691a13a7e5698a5b12e" candidate_uuid: "d7a6f415051b487eaefa063e0bb43960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" is_pre_election: true
I20260812 11:10:08.482919 29930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9153228cffb94691a13a7e5698a5b12e" candidate_uuid: "d7a6f415051b487eaefa063e0bb43960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03be7361f2d54061897ab2be44de9a2d" is_pre_election: true
I20260812 11:10:08.483106 30161 raft_consensus.cc:2454] T 9153228cffb94691a13a7e5698a5b12e P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7a6f415051b487eaefa063e0bb43960 in term 0.
I20260812 11:10:08.483140 29930 raft_consensus.cc:2454] T 9153228cffb94691a13a7e5698a5b12e P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7a6f415051b487eaefa063e0bb43960 in term 0.
I20260812 11:10:08.483453 29621 leader_election.cc:302] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03be7361f2d54061897ab2be44de9a2d, d7a6f415051b487eaefa063e0bb43960; no voters:
I20260812 11:10:08.483672 30284 raft_consensus.cc:2781] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:10:08.483747 30284 raft_consensus.cc:482] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:10:08.483800 30284 raft_consensus.cc:3037] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:08.484799 30284 raft_consensus.cc:504] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.485116 30284 leader_election.cc:288] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960 [CANDIDATE]: Term 1 election: Requested vote from peers 03be7361f2d54061897ab2be44de9a2d (127.17.44.194:43485), 82875c708dc345eba57b0ea8d0a4dc3f (127.17.44.195:46469)
I20260812 11:10:08.485569 30161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9153228cffb94691a13a7e5698a5b12e" candidate_uuid: "d7a6f415051b487eaefa063e0bb43960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82875c708dc345eba57b0ea8d0a4dc3f"
I20260812 11:10:08.485595 29930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9153228cffb94691a13a7e5698a5b12e" candidate_uuid: "d7a6f415051b487eaefa063e0bb43960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03be7361f2d54061897ab2be44de9a2d"
I20260812 11:10:08.485699 30161 raft_consensus.cc:3037] T 9153228cffb94691a13a7e5698a5b12e P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:08.485710 29930 raft_consensus.cc:3037] T 9153228cffb94691a13a7e5698a5b12e P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:08.486594 30161 raft_consensus.cc:2454] T 9153228cffb94691a13a7e5698a5b12e P 82875c708dc345eba57b0ea8d0a4dc3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7a6f415051b487eaefa063e0bb43960 in term 1.
I20260812 11:10:08.486794 29930 raft_consensus.cc:2454] T 9153228cffb94691a13a7e5698a5b12e P 03be7361f2d54061897ab2be44de9a2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7a6f415051b487eaefa063e0bb43960 in term 1.
I20260812 11:10:08.486909 29621 leader_election.cc:302] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82875c708dc345eba57b0ea8d0a4dc3f, d7a6f415051b487eaefa063e0bb43960; no voters:
I20260812 11:10:08.487146 30284 raft_consensus.cc:2781] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:10:08.487273 30284 raft_consensus.cc:686] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960 [term 1 LEADER]: Becoming Leader. State: Replica: d7a6f415051b487eaefa063e0bb43960, State: Running, Role: LEADER
I20260812 11:10:08.487430 30284 consensus_queue.cc:237] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.489423 29478 catalog_manager.cc:5495] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7a6f415051b487eaefa063e0bb43960 (127.17.44.193). New cstate: current_term: 1 leader_uuid: "d7a6f415051b487eaefa063e0bb43960" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } health_report { overall_health: UNKNOWN } } }
I20260812 11:10:08.510890 30263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:08.507374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55970 (local address 127.17.44.193:39365)
0812 11:10:08.507552 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:08.507556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:08.507582 (+ 26us) server_negotiation.cc:407] Connection header received
0812 11:10:08.507781 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:08.507784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:08.507844 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:08.507923 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:10:08.508248 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.508771 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.509776 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.510006 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.510124 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:08.510417 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:10:08.510491 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:10:08.510575 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:08.510675 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 11:10:08.510733 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 11:10:08.513135 29930 raft_consensus.cc:1261] T 9153228cffb94691a13a7e5698a5b12e P 03be7361f2d54061897ab2be44de9a2d [term 1 FOLLOWER]: Refusing update from remote peer d7a6f415051b487eaefa063e0bb43960: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:10:08.513293 30161 raft_consensus.cc:1261] T 9153228cffb94691a13a7e5698a5b12e P 82875c708dc345eba57b0ea8d0a4dc3f [term 1 FOLLOWER]: Refusing update from remote peer d7a6f415051b487eaefa063e0bb43960: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:10:08.513608 30284 consensus_queue.cc:1035] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:10:08.514154 30347 consensus_queue.cc:1035] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:10:08.521512 29484 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:10:08.521665 29484 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:10:08.524083 29484 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c: Sending DeleteTablet for 3 replicas of tablet 9153228cffb94691a13a7e5698a5b12e
I20260812 11:10:08.524708 29700 tablet_service.cc:1514] Processing DeleteTablet for tablet 9153228cffb94691a13a7e5698a5b12e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:10:08 UTC) from {username='oryx', principal='oryx/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:55958
I20260812 11:10:08.524896 29915 tablet_service.cc:1514] Processing DeleteTablet for tablet 9153228cffb94691a13a7e5698a5b12e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:10:08 UTC) from {username='oryx', principal='oryx/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:53518
I20260812 11:10:08.525103 30305 tablet_replica.cc:330] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960: stopping tablet replica
I20260812 11:10:08.525321 30306 tablet_replica.cc:330] T 9153228cffb94691a13a7e5698a5b12e P 03be7361f2d54061897ab2be44de9a2d: stopping tablet replica
I20260812 11:10:08.525146 30124 tablet_service.cc:1514] Processing DeleteTablet for tablet 9153228cffb94691a13a7e5698a5b12e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:10:08 UTC) from {username='oryx', principal='oryx/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:42292
I20260812 11:10:08.525445 30306 raft_consensus.cc:2229] T 9153228cffb94691a13a7e5698a5b12e P 03be7361f2d54061897ab2be44de9a2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:10:08.525458 30305 raft_consensus.cc:2229] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:10:08.525590 30306 raft_consensus.cc:2258] T 9153228cffb94691a13a7e5698a5b12e P 03be7361f2d54061897ab2be44de9a2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:10:08.525646 30305 raft_consensus.cc:2258] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:10:08.525689 29484 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39424:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 11:10:08.525847 30307 tablet_replica.cc:330] T 9153228cffb94691a13a7e5698a5b12e P 82875c708dc345eba57b0ea8d0a4dc3f: stopping tablet replica
I20260812 11:10:08.526018 30307 raft_consensus.cc:2229] T 9153228cffb94691a13a7e5698a5b12e P 82875c708dc345eba57b0ea8d0a4dc3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:10:08.526139 30307 raft_consensus.cc:2258] T 9153228cffb94691a13a7e5698a5b12e P 82875c708dc345eba57b0ea8d0a4dc3f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 11:10:08.526126 29484 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:10:08.526291 30306 ts_tablet_manager.cc:1905] T 9153228cffb94691a13a7e5698a5b12e P 03be7361f2d54061897ab2be44de9a2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:10:08.526906 30307 ts_tablet_manager.cc:1905] T 9153228cffb94691a13a7e5698a5b12e P 82875c708dc345eba57b0ea8d0a4dc3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:10:08.527503 30305 ts_tablet_manager.cc:1905] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:10:08.529011 30306 ts_tablet_manager.cc:1918] T 9153228cffb94691a13a7e5698a5b12e P 03be7361f2d54061897ab2be44de9a2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:10:08.529093 30306 log.cc:1196] T 9153228cffb94691a13a7e5698a5b12e P 03be7361f2d54061897ab2be44de9a2d: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/9153228cffb94691a13a7e5698a5b12e
I20260812 11:10:08.529392 30306 ts_tablet_manager.cc:1939] T 9153228cffb94691a13a7e5698a5b12e P 03be7361f2d54061897ab2be44de9a2d: Deleting consensus metadata
I20260812 11:10:08.530519 30305 ts_tablet_manager.cc:1918] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:10:08.530618 30305 log.cc:1196] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-0/wal/wals/9153228cffb94691a13a7e5698a5b12e
I20260812 11:10:08.530966 30305 ts_tablet_manager.cc:1939] T 9153228cffb94691a13a7e5698a5b12e P d7a6f415051b487eaefa063e0bb43960: Deleting consensus metadata
I20260812 11:10:08.531059 29466 catalog_manager.cc:4841] TS 03be7361f2d54061897ab2be44de9a2d (127.17.44.194:43485): tablet 9153228cffb94691a13a7e5698a5b12e (table test-table [id=4f31833bb73547fba2c17ea3da335032]) successfully deleted
I20260812 11:10:08.531715 29466 catalog_manager.cc:4841] TS d7a6f415051b487eaefa063e0bb43960 (127.17.44.193:39365): tablet 9153228cffb94691a13a7e5698a5b12e (table test-table [id=4f31833bb73547fba2c17ea3da335032]) successfully deleted
I20260812 11:10:08.531747 30307 ts_tablet_manager.cc:1918] T 9153228cffb94691a13a7e5698a5b12e P 82875c708dc345eba57b0ea8d0a4dc3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:10:08.532056 30307 log.cc:1196] T 9153228cffb94691a13a7e5698a5b12e P 82875c708dc345eba57b0ea8d0a4dc3f: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/9153228cffb94691a13a7e5698a5b12e
I20260812 11:10:08.532358 30307 ts_tablet_manager.cc:1939] T 9153228cffb94691a13a7e5698a5b12e P 82875c708dc345eba57b0ea8d0a4dc3f: Deleting consensus metadata
I20260812 11:10:08.533332 29466 catalog_manager.cc:4841] TS 82875c708dc345eba57b0ea8d0a4dc3f (127.17.44.195:46469): tablet 9153228cffb94691a13a7e5698a5b12e (table test-table [id=4f31833bb73547fba2c17ea3da335032]) successfully deleted
I20260812 11:10:08.534397 29700 tablet_service.cc:1467] Processing CreateTablet for tablet ae83c2a85e134b338ea92b792e69b4e6 (DEFAULT_TABLE table=test-table [id=ff72b383c6f842bd8e41e9c5db7cf8e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:10:08.534533 30124 tablet_service.cc:1467] Processing CreateTablet for tablet ae83c2a85e134b338ea92b792e69b4e6 (DEFAULT_TABLE table=test-table [id=ff72b383c6f842bd8e41e9c5db7cf8e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:10:08.534662 29700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae83c2a85e134b338ea92b792e69b4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:08.534756 30124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae83c2a85e134b338ea92b792e69b4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:08.536469 30267 tablet_bootstrap.cc:492] T ae83c2a85e134b338ea92b792e69b4e6 P d7a6f415051b487eaefa063e0bb43960: Bootstrap starting.
I20260812 11:10:08.536851 29915 tablet_service.cc:1467] Processing CreateTablet for tablet ae83c2a85e134b338ea92b792e69b4e6 (DEFAULT_TABLE table=test-table [id=ff72b383c6f842bd8e41e9c5db7cf8e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:10:08.537128 29915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae83c2a85e134b338ea92b792e69b4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:08.537300 30267 tablet_bootstrap.cc:654] T ae83c2a85e134b338ea92b792e69b4e6 P d7a6f415051b487eaefa063e0bb43960: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:08.537803 30264 tablet_bootstrap.cc:492] T ae83c2a85e134b338ea92b792e69b4e6 P 82875c708dc345eba57b0ea8d0a4dc3f: Bootstrap starting.
I20260812 11:10:08.538425 30267 tablet_bootstrap.cc:492] T ae83c2a85e134b338ea92b792e69b4e6 P d7a6f415051b487eaefa063e0bb43960: No bootstrap required, opened a new log
I20260812 11:10:08.538527 30267 ts_tablet_manager.cc:1397] T ae83c2a85e134b338ea92b792e69b4e6 P d7a6f415051b487eaefa063e0bb43960: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 11:10:08.538676 30264 tablet_bootstrap.cc:654] T ae83c2a85e134b338ea92b792e69b4e6 P 82875c708dc345eba57b0ea8d0a4dc3f: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:08.538983 30267 raft_consensus.cc:348] T ae83c2a85e134b338ea92b792e69b4e6 P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.539121 30267 raft_consensus.cc:374] T ae83c2a85e134b338ea92b792e69b4e6 P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:08.539160 30267 raft_consensus.cc:729] T ae83c2a85e134b338ea92b792e69b4e6 P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7a6f415051b487eaefa063e0bb43960, State: Initialized, Role: FOLLOWER
I20260812 11:10:08.539286 30267 consensus_queue.cc:260] T ae83c2a85e134b338ea92b792e69b4e6 P d7a6f415051b487eaefa063e0bb43960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.539605 30267 ts_tablet_manager.cc:1428] T ae83c2a85e134b338ea92b792e69b4e6 P d7a6f415051b487eaefa063e0bb43960: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 11:10:08.539716 30270 tablet_bootstrap.cc:492] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d: Bootstrap starting.
I20260812 11:10:08.539754 30264 tablet_bootstrap.cc:492] T ae83c2a85e134b338ea92b792e69b4e6 P 82875c708dc345eba57b0ea8d0a4dc3f: No bootstrap required, opened a new log
I20260812 11:10:08.539830 30264 ts_tablet_manager.cc:1397] T ae83c2a85e134b338ea92b792e69b4e6 P 82875c708dc345eba57b0ea8d0a4dc3f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 11:10:08.540263 30264 raft_consensus.cc:348] T ae83c2a85e134b338ea92b792e69b4e6 P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.540374 30264 raft_consensus.cc:374] T ae83c2a85e134b338ea92b792e69b4e6 P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:08.540416 30264 raft_consensus.cc:729] T ae83c2a85e134b338ea92b792e69b4e6 P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82875c708dc345eba57b0ea8d0a4dc3f, State: Initialized, Role: FOLLOWER
I20260812 11:10:08.540539 30264 consensus_queue.cc:260] T ae83c2a85e134b338ea92b792e69b4e6 P 82875c708dc345eba57b0ea8d0a4dc3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.540658 30270 tablet_bootstrap.cc:654] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:08.540827 30264 ts_tablet_manager.cc:1428] T ae83c2a85e134b338ea92b792e69b4e6 P 82875c708dc345eba57b0ea8d0a4dc3f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 11:10:08.541913 30270 tablet_bootstrap.cc:492] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d: No bootstrap required, opened a new log
I20260812 11:10:08.542014 30270 ts_tablet_manager.cc:1397] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 11:10:08.542477 30270 raft_consensus.cc:348] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.542559 30270 raft_consensus.cc:374] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:08.542580 30270 raft_consensus.cc:729] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03be7361f2d54061897ab2be44de9a2d, State: Initialized, Role: FOLLOWER
I20260812 11:10:08.542713 30270 consensus_queue.cc:260] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.543079 30270 ts_tablet_manager.cc:1428] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:10:08.726635 30279 raft_consensus.cc:482] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:10:08.726827 30279 raft_consensus.cc:504] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.728341 30279 leader_election.cc:288] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7a6f415051b487eaefa063e0bb43960 (127.17.44.193:39365), 82875c708dc345eba57b0ea8d0a4dc3f (127.17.44.195:46469)
I20260812 11:10:08.734601 30263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:08.728787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:47813 (local address 127.17.44.193:39365)
0812 11:10:08.728953 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:08.728957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:08.729135 (+ 178us) server_negotiation.cc:407] Connection header received
0812 11:10:08.729477 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:08.729481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:08.729539 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:08.729649 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:08.730191 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.731016 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.733113 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.733725 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.733855 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:08.733916 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:08.734382 (+ 466us) server_negotiation.cc:299] Negotiation successful
0812 11:10:08.734461 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 11:10:08.734601 30261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:08.729327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:41243 (local address 127.17.44.195:46469)
0812 11:10:08.729455 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:08.729459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:08.729475 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:10:08.729593 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:08.729599 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:08.729653 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:08.729725 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:08.730191 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.731016 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.733113 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.733725 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.733856 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:08.733924 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:08.734381 (+ 457us) server_negotiation.cc:299] Negotiation successful
0812 11:10:08.734459 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 11:10:08.734683 30388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:08.729127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:46469 (local address 127.17.44.194:41243)
0812 11:10:08.729381 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:08.729392 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:08.729462 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:08.729734 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:08.729737 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:08.729792 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:08.730035 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:08.730041 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.731176 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.731181 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:08.732943 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:08.732957 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.733884 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.733889 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:08.734129 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:08.734235 (+ 106us) client_negotiation.cc:769] Sending connection context
0812 11:10:08.734321 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 11:10:08.734450 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":323,"spinlock_wait_cycles":13440,"thread_start_us":80,"threads_started":1}
I20260812 11:10:08.734705 30386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:08.728654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:39365 (local address 127.17.44.194:47813)
0812 11:10:08.728966 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:08.728984 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:08.729141 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:08.729712 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:08.729717 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:08.729791 (+ 74us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:08.730018 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:08.730027 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.731176 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.731181 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:08.732943 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:08.732957 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.733884 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.733889 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:08.734127 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:08.734224 (+ 97us) client_negotiation.cc:769] Sending connection context
0812 11:10:08.734321 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 11:10:08.734450 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":269,"spinlock_wait_cycles":48512,"thread_start_us":132,"threads_started":1}
I20260812 11:10:08.735253 30161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae83c2a85e134b338ea92b792e69b4e6" candidate_uuid: "03be7361f2d54061897ab2be44de9a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" is_pre_election: true
I20260812 11:10:08.735327 29727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae83c2a85e134b338ea92b792e69b4e6" candidate_uuid: "03be7361f2d54061897ab2be44de9a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7a6f415051b487eaefa063e0bb43960" is_pre_election: true
I20260812 11:10:08.735426 30161 raft_consensus.cc:2454] T ae83c2a85e134b338ea92b792e69b4e6 P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03be7361f2d54061897ab2be44de9a2d in term 0.
I20260812 11:10:08.735512 29727 raft_consensus.cc:2454] T ae83c2a85e134b338ea92b792e69b4e6 P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03be7361f2d54061897ab2be44de9a2d in term 0.
I20260812 11:10:08.735850 29840 leader_election.cc:302] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03be7361f2d54061897ab2be44de9a2d, 82875c708dc345eba57b0ea8d0a4dc3f; no voters:
I20260812 11:10:08.736243 30279 raft_consensus.cc:2781] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:10:08.736366 30279 raft_consensus.cc:482] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:10:08.736424 30279 raft_consensus.cc:3037] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:08.737888 30279 raft_consensus.cc:504] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.738322 30279 leader_election.cc:288] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d [CANDIDATE]: Term 1 election: Requested vote from peers d7a6f415051b487eaefa063e0bb43960 (127.17.44.193:39365), 82875c708dc345eba57b0ea8d0a4dc3f (127.17.44.195:46469)
I20260812 11:10:08.738724 29727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae83c2a85e134b338ea92b792e69b4e6" candidate_uuid: "03be7361f2d54061897ab2be44de9a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7a6f415051b487eaefa063e0bb43960"
I20260812 11:10:08.738878 29727 raft_consensus.cc:3037] T ae83c2a85e134b338ea92b792e69b4e6 P d7a6f415051b487eaefa063e0bb43960 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:08.738833 30161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae83c2a85e134b338ea92b792e69b4e6" candidate_uuid: "03be7361f2d54061897ab2be44de9a2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82875c708dc345eba57b0ea8d0a4dc3f"
I20260812 11:10:08.738999 30161 raft_consensus.cc:3037] T ae83c2a85e134b338ea92b792e69b4e6 P 82875c708dc345eba57b0ea8d0a4dc3f [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:08.739912 29727 raft_consensus.cc:2454] T ae83c2a85e134b338ea92b792e69b4e6 P d7a6f415051b487eaefa063e0bb43960 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03be7361f2d54061897ab2be44de9a2d in term 1.
I20260812 11:10:08.740028 30161 raft_consensus.cc:2454] T ae83c2a85e134b338ea92b792e69b4e6 P 82875c708dc345eba57b0ea8d0a4dc3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03be7361f2d54061897ab2be44de9a2d in term 1.
I20260812 11:10:08.740250 29840 leader_election.cc:302] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03be7361f2d54061897ab2be44de9a2d, d7a6f415051b487eaefa063e0bb43960; no voters:
I20260812 11:10:08.740453 30279 raft_consensus.cc:2781] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:10:08.740680 30279 raft_consensus.cc:686] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d [term 1 LEADER]: Becoming Leader. State: Replica: 03be7361f2d54061897ab2be44de9a2d, State: Running, Role: LEADER
I20260812 11:10:08.740932 30279 consensus_queue.cc:237] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } }
I20260812 11:10:08.743613 29484 catalog_manager.cc:5495] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 03be7361f2d54061897ab2be44de9a2d (127.17.44.194). New cstate: current_term: 1 leader_uuid: "03be7361f2d54061897ab2be44de9a2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03be7361f2d54061897ab2be44de9a2d" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 43485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 } health_report { overall_health: UNKNOWN } } }
I20260812 11:10:08.786360 30394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:08.782365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53534 (local address 127.17.44.194:43485)
0812 11:10:08.782784 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:08.782791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:08.782803 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:10:08.782872 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:08.782874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:08.782975 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:08.783119 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:10:08.783369 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.783921 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.784747 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.784997 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.785107 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:08.785403 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:10:08.785882 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:10:08.786067 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:08.786133 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 11:10:08.786187 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":184,"threads_started":1}
I20260812 11:10:08.797873 30261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:08.794054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:36981 (local address 127.17.44.195:46469)
0812 11:10:08.794175 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:08.794178 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:08.794592 (+ 414us) server_negotiation.cc:407] Connection header received
0812 11:10:08.794782 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:08.794785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:08.794832 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:08.794923 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:10:08.795329 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.795811 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.796716 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.796941 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.797041 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:08.797384 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:10:08.797477 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:10:08.797548 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:08.797717 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 11:10:08.797763 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 11:10:08.797887 30398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:08.793901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:46469 (local address 127.17.44.194:36981)
0812 11:10:08.794437 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:08.794457 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:08.794619 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:08.795049 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:08.795053 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:08.795063 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 11:10:08.795189 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:08.795194 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.795932 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.795935 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:08.796585 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:08.796594 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.797090 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.797093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:08.797168 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:08.797213 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 11:10:08.797592 (+ 379us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 11:10:08.797604 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 11:10:08.797703 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 11:10:08.797766 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":259,"threads_started":1}
I20260812 11:10:08.804594 30161 raft_consensus.cc:1261] T ae83c2a85e134b338ea92b792e69b4e6 P 82875c708dc345eba57b0ea8d0a4dc3f [term 1 FOLLOWER]: Refusing update from remote peer 03be7361f2d54061897ab2be44de9a2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:10:08.804641 29727 raft_consensus.cc:1261] T ae83c2a85e134b338ea92b792e69b4e6 P d7a6f415051b487eaefa063e0bb43960 [term 1 FOLLOWER]: Refusing update from remote peer 03be7361f2d54061897ab2be44de9a2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:10:08.805123 30391 consensus_queue.cc:1035] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "82875c708dc345eba57b0ea8d0a4dc3f" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:10:08.805366 30279 consensus_queue.cc:1035] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7a6f415051b487eaefa063e0bb43960" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:10:08.826654 30352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:08.822836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:43485 (local address 127.17.44.195:52321)
0812 11:10:08.823007 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:08.823023 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:08.823114 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:08.823498 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:08.823505 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:08.823515 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 11:10:08.823634 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:08.823641 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.824477 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.824480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:08.825042 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:08.825051 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.825968 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.825970 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:08.826034 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:08.826060 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 11:10:08.826352 (+ 292us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 11:10:08.826362 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 11:10:08.826455 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 11:10:08.826546 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 11:10:08.826646 30394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:08.823095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:52321 (local address 127.17.44.194:43485)
0812 11:10:08.823217 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:08.823221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:08.823241 (+ 20us) server_negotiation.cc:407] Connection header received
0812 11:10:08.823299 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:08.823302 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:08.823350 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:08.823435 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:10:08.823825 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.824339 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.825328 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:08.825590 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:08.825730 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:08.826204 (+ 474us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:10:08.826268 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:10:08.826348 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:08.826467 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 11:10:08.826516 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 11:10:08.853519 29478 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:10:08.853709 29478 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:10:08.856598 29478 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 977cbb4bc25242a8965c95da8803335c: Sending DeleteTablet for 3 replicas of tablet ae83c2a85e134b338ea92b792e69b4e6
I20260812 11:10:08.857451 29915 tablet_service.cc:1514] Processing DeleteTablet for tablet ae83c2a85e134b338ea92b792e69b4e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:10:08 UTC) from {username='oryx', principal='oryx/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:53518
I20260812 11:10:08.857645 30306 tablet_replica.cc:330] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d: stopping tablet replica
I20260812 11:10:08.857779 30306 raft_consensus.cc:2229] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:10:08.857823 29700 tablet_service.cc:1514] Processing DeleteTablet for tablet ae83c2a85e134b338ea92b792e69b4e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:10:08 UTC) from {username='oryx', principal='oryx/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:55958
I20260812 11:10:08.858079 30306 raft_consensus.cc:2258] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:10:08.858155 30305 tablet_replica.cc:330] T ae83c2a85e134b338ea92b792e69b4e6 P d7a6f415051b487eaefa063e0bb43960: stopping tablet replica
I20260812 11:10:08.858312 30305 raft_consensus.cc:2229] T ae83c2a85e134b338ea92b792e69b4e6 P d7a6f415051b487eaefa063e0bb43960 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:10:08.858353 30124 tablet_service.cc:1514] Processing DeleteTablet for tablet ae83c2a85e134b338ea92b792e69b4e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:10:08 UTC) from {username='oryx', principal='oryx/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:42292
I20260812 11:10:08.858474 30305 raft_consensus.cc:2258] T ae83c2a85e134b338ea92b792e69b4e6 P d7a6f415051b487eaefa063e0bb43960 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:10:08.858558 30307 tablet_replica.cc:330] T ae83c2a85e134b338ea92b792e69b4e6 P 82875c708dc345eba57b0ea8d0a4dc3f: stopping tablet replica
I20260812 11:10:08.858690 30307 raft_consensus.cc:2229] T ae83c2a85e134b338ea92b792e69b4e6 P 82875c708dc345eba57b0ea8d0a4dc3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:10:08.858819 30307 raft_consensus.cc:2258] T ae83c2a85e134b338ea92b792e69b4e6 P 82875c708dc345eba57b0ea8d0a4dc3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:10:08.859961 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 29593
I20260812 11:10:08.860354 30306 ts_tablet_manager.cc:1905] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:10:08.860356 30307 ts_tablet_manager.cc:1905] T ae83c2a85e134b338ea92b792e69b4e6 P 82875c708dc345eba57b0ea8d0a4dc3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:10:08.862874 30307 ts_tablet_manager.cc:1918] T ae83c2a85e134b338ea92b792e69b4e6 P 82875c708dc345eba57b0ea8d0a4dc3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:10:08.862874 30306 ts_tablet_manager.cc:1918] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 11:10:08.862975 30306 log.cc:1196] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/ae83c2a85e134b338ea92b792e69b4e6
I20260812 11:10:08.863279 30306 ts_tablet_manager.cc:1939] T ae83c2a85e134b338ea92b792e69b4e6 P 03be7361f2d54061897ab2be44de9a2d: Deleting consensus metadata
I20260812 11:10:08.865274 30307 log.cc:1196] T ae83c2a85e134b338ea92b792e69b4e6 P 82875c708dc345eba57b0ea8d0a4dc3f: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/ae83c2a85e134b338ea92b792e69b4e6
I20260812 11:10:08.865705 30307 ts_tablet_manager.cc:1939] T ae83c2a85e134b338ea92b792e69b4e6 P 82875c708dc345eba57b0ea8d0a4dc3f: Deleting consensus metadata
I20260812 11:10:08.866343 29466 catalog_manager.cc:4841] TS 82875c708dc345eba57b0ea8d0a4dc3f (127.17.44.195:46469): tablet ae83c2a85e134b338ea92b792e69b4e6 (table test-table [id=ff72b383c6f842bd8e41e9c5db7cf8e7]) successfully deleted
I20260812 11:10:08.867199 29466 catalog_manager.cc:4841] TS 03be7361f2d54061897ab2be44de9a2d (127.17.44.194:43485): tablet ae83c2a85e134b338ea92b792e69b4e6 (table test-table [id=ff72b383c6f842bd8e41e9c5db7cf8e7]) successfully deleted
W20260812 11:10:08.876744 29466 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.44.193:39365 (error 108)
I20260812 11:10:08.877813 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 29814
I20260812 11:10:08.882062 30415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:08.881558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:39365 (local address 127.0.0.1:55984)
0812 11:10:08.881900 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:08.881970 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.193:39365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":71,"threads_started":1}
W20260812 11:10:08.882292 29466 catalog_manager.cc:4584] TS d7a6f415051b487eaefa063e0bb43960 (127.17.44.193:39365): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ae83c2a85e134b338ea92b792e69b4e6: Network error: Client connection negotiation failed: client connection to 127.17.44.193:39365: connect: Connection refused (error 111)
I20260812 11:10:08.887988 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 30026
I20260812 11:10:08.908396 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 29433
2026-08-12T11:10:08Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3733 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:10:08 dist-test-slave-drl0 krb5kdc[30419](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:10:08 dist-test-slave-drl0 krb5kdc[30419](info): set up 2 sockets
Aug 12 11:10:08 dist-test-slave-drl0 krb5kdc[30419](info): commencing operation
krb5kdc: starting...
W20260812 11:10:10.233705 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:10:10 dist-test-slave-drl0 krb5kdc[30419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:10:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:10:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:10:10.370747 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:44053
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:43379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:44053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:10:10.488482 30434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:10.488782 30434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:10.488859 30434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:10.492897 30434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:10:10.492986 30434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:10.493021 30434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:10.493077 30434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:10:10.493113 30434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:10:10.498391 30434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:43379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:44053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:44053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:10.499569 30434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:10.500638 30434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:10.510620 30444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:10.510572 30443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:10.510902 30434 server_base.cc:1034] running on GCE node
W20260812 11:10:10.510608 30447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:10.511719 30434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:10.512779 30434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:10.513981 30434 hybrid_clock.cc:648] HybridClock initialized: now 1786533010513949 us; error 44 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T11:10:10Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1605 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:10:10 dist-test-slave-drl0 krb5kdc[30458](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:10:10 dist-test-slave-drl0 krb5kdc[30458](info): set up 2 sockets
Aug 12 11:10:10 dist-test-slave-drl0 krb5kdc[30458](info): commencing operation
krb5kdc: starting...
W20260812 11:10:11.831522 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.280s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:10:11 dist-test-slave-drl0 krb5kdc[30458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:10:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:10:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:10:11.976054 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:42379
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:42669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:42379
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:10:12.097569 30472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:12.097828 30472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:12.097868 30472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:12.101804 30472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:10:12.101858 30472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:12.101878 30472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:12.101894 30472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:10:12.101909 30472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:10:12.107085 30472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:42669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:42379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:42379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:12.108047 30472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:12.108994 30472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:12.118011 30477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:12.118006 30479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:12.118005 30481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:12.118458 30472 server_base.cc:1034] running on GCE node
I20260812 11:10:12.119001 30472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:12.120054 30472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:12.121238 30472 hybrid_clock.cc:648] HybridClock initialized: now 1786533012121222 us; error 34 us; skew 500 ppm
Aug 12 11:10:12 dist-test-slave-drl0 krb5kdc[30458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533012, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:10:12.127297 30472 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:10:12.128167 30472 webserver.cc:460] Webserver started at http://127.17.44.254:36493/ using document root <none> and password file <none>
I20260812 11:10:12.128739 30472 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:12.128804 30472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:12.129019 30472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:10:12.131162 30472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "81b078f8a5454db5abbc2804776ae1d9"
format_stamp: "Formatted at 2026-08-12 11:10:12 on dist-test-slave-drl0"
server_key: "87e9b2bb7e2fd051d27b36f234fe6f2d"
server_key_iv: "d81db2607eb7e4d28cf2d2442de7bd27"
server_key_version: "encryptionkey@0"
I20260812 11:10:12.131759 30472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "81b078f8a5454db5abbc2804776ae1d9"
format_stamp: "Formatted at 2026-08-12 11:10:12 on dist-test-slave-drl0"
server_key: "87e9b2bb7e2fd051d27b36f234fe6f2d"
server_key_iv: "d81db2607eb7e4d28cf2d2442de7bd27"
server_key_version: "encryptionkey@0"
I20260812 11:10:12.136296 30472 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 11:10:12.139568 30491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:12.141301 30472 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:10:12.141629 30472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "81b078f8a5454db5abbc2804776ae1d9"
format_stamp: "Formatted at 2026-08-12 11:10:12 on dist-test-slave-drl0"
server_key: "87e9b2bb7e2fd051d27b36f234fe6f2d"
server_key_iv: "d81db2607eb7e4d28cf2d2442de7bd27"
server_key_version: "encryptionkey@0"
I20260812 11:10:12.141793 30472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:12.159483 30472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:12.163110 30472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:12.163349 30472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:12.171343 30472 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:42379
I20260812 11:10:12.171397 30571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:42379 every 8 connection(s)
I20260812 11:10:12.172483 30472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:10:12.175485 30572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:12.175647 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 30472
I20260812 11:10:12.175784 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:10:12.176117 17587 external_mini_cluster.cc:1428] Setting key adc398915405fa7bf8511cd81ed44507
I20260812 11:10:12.182200 30572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9: Bootstrap starting.
I20260812 11:10:12.184979 30572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:12.185773 30572 log.cc:824] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9: Log is configured to *not* fsync() on all Append() calls
Aug 12 11:10:12 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:10:12.187841 30572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9: No bootstrap required, opened a new log
I20260812 11:10:12.192291 30572 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b078f8a5454db5abbc2804776ae1d9" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42379 } }
I20260812 11:10:12.192574 30572 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:12.192627 30572 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81b078f8a5454db5abbc2804776ae1d9, State: Initialized, Role: FOLLOWER
I20260812 11:10:12.193145 30572 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b078f8a5454db5abbc2804776ae1d9" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42379 } }
I20260812 11:10:12.193329 30572 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:10:12.193442 30572 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:10:12.193567 30572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:12.194552 30572 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b078f8a5454db5abbc2804776ae1d9" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42379 } }
I20260812 11:10:12.194972 30572 leader_election.cc:302] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81b078f8a5454db5abbc2804776ae1d9; no voters:
I20260812 11:10:12.195297 30575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:12.177554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60536 (local address 127.17.44.254:42379)
0812 11:10:12.178360 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:12.178383 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:12.178444 (+ 61us) server_negotiation.cc:407] Connection header received
0812 11:10:12.179589 (+ 1145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:12.179622 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:12.180097 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:12.180482 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:12.181549 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:12.183190 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:12.184672 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:12.185298 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:12.185488 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:12.188225 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:12.188232 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:12.188252 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:12.188283 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:12.191113 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:12.191618 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:12.191626 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:12.191633 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:12.191776 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:12.192098 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:12.192102 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:12.192104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:12.192668 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:12.192877 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:12.193411 (+ 534us) server_negotiation.cc:299] Negotiation successful
0812 11:10:12.193947 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":137,"threads_started":1}
I20260812 11:10:12.195367 30572 leader_election.cc:288] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:10:12.195443 30577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:10:12.195652 30577 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [term 1 LEADER]: Becoming Leader. State: Replica: 81b078f8a5454db5abbc2804776ae1d9, State: Running, Role: LEADER
I20260812 11:10:12.195973 30577 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b078f8a5454db5abbc2804776ae1d9" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42379 } }
I20260812 11:10:12.196696 30572 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:10:12.199501 30584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81b078f8a5454db5abbc2804776ae1d9. Latest consensus state: current_term: 1 leader_uuid: "81b078f8a5454db5abbc2804776ae1d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b078f8a5454db5abbc2804776ae1d9" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42379 } } }
I20260812 11:10:12.199528 30586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81b078f8a5454db5abbc2804776ae1d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b078f8a5454db5abbc2804776ae1d9" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42379 } } }
I20260812 11:10:12.199631 30584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [sys.catalog]: This master's current role is: LEADER
I20260812 11:10:12.199649 30586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [sys.catalog]: This master's current role is: LEADER
W20260812 11:10:12.201736 30604 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:10:12.201850 30604 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:10:12.201932 30607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:10:12.202623 30607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:10:12.208392 30607 catalog_manager.cc:1347] Generated new cluster ID: 42218549de974fcf8d4d41fc1f3d89da
I20260812 11:10:12.208465 30607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:10:12.227464 30607 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:10:12.228443 30607 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:10:12.242796 30607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9: Generated new TSK 0
I20260812 11:10:12.243564 30607 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
I20260812 11:10:12.286705 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:42379
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:42669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:10:12.413111 30616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:12.413429 30616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:12.413522 30616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:12.417579 30616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:12.417673 30616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:12.417769 30616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:10:12.423341 30616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:42669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:42379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:12.424398 30616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:12.425412 30616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:12.435494 30622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:12.435580 30625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:12.435688 30623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:12.435967 30616 server_base.cc:1034] running on GCE node
I20260812 11:10:12.436367 30616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:12.437127 30616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:12.438355 30616 hybrid_clock.cc:648] HybridClock initialized: now 1786533012438317 us; error 53 us; skew 500 ppm
Aug 12 11:10:12 dist-test-slave-drl0 krb5kdc[30458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533012, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:10:12.441351 30616 init.cc:377] Logged in from keytab as kudu/127.17.44.193@KRBTEST.COM (short username kudu)
I20260812 11:10:12.442221 30616 webserver.cc:460] Webserver started at http://127.17.44.193:37289/ using document root <none> and password file <none>
I20260812 11:10:12.442791 30616 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:12.442883 30616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:12.443148 30616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:10:12.446162 30616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "240e712ea353491986889786e0e3ce3b"
format_stamp: "Formatted at 2026-08-12 11:10:12 on dist-test-slave-drl0"
server_key: "8db9fb01c3ad368e834883ec7f12ec4f"
server_key_iv: "3b4099b260ce2a86a143f8a9c53ec8f8"
server_key_version: "encryptionkey@0"
I20260812 11:10:12.446866 30616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "240e712ea353491986889786e0e3ce3b"
format_stamp: "Formatted at 2026-08-12 11:10:12 on dist-test-slave-drl0"
server_key: "8db9fb01c3ad368e834883ec7f12ec4f"
server_key_iv: "3b4099b260ce2a86a143f8a9c53ec8f8"
server_key_version: "encryptionkey@0"
I20260812 11:10:12.450894 30616 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 11:10:12.453647 30635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:12.455288 30616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:10:12.455438 30616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "240e712ea353491986889786e0e3ce3b"
format_stamp: "Formatted at 2026-08-12 11:10:12 on dist-test-slave-drl0"
server_key: "8db9fb01c3ad368e834883ec7f12ec4f"
server_key_iv: "3b4099b260ce2a86a143f8a9c53ec8f8"
server_key_version: "encryptionkey@0"
I20260812 11:10:12.455557 30616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:12.478407 30616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:12.481822 30616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:12.482069 30616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:12.482695 30616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:10:12.483590 30616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:10:12.483667 30616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:12.483740 30616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:10:12.483793 30616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:12.494694 30616 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:40873
I20260812 11:10:12.494724 30809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:40873 every 8 connection(s)
I20260812 11:10:12.495709 30616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:10:12.505815 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 30616
I20260812 11:10:12.505993 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:10:12.506320 17587 external_mini_cluster.cc:1428] Setting key a793d12be9871ca4a962a9c65538c665
Aug 12 11:10:12 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533012, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:10:12.512250 30575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:12.498393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:43769 (local address 127.17.44.254:42379)
0812 11:10:12.498564 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:12.498568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:12.499875 (+ 1307us) server_negotiation.cc:407] Connection header received
0812 11:10:12.500789 (+ 914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:12.500882 (+ 93us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:12.500971 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:12.501094 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:12.502280 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:12.502838 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:12.504122 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:12.504365 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:12.504484 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:12.507188 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:12.507193 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:12.507196 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:12.507223 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:12.509211 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:12.510069 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:12.510077 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:12.510081 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:12.510223 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:12.510700 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:12.510706 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:12.510711 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:12.511059 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:12.511178 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:12.512005 (+ 827us) server_negotiation.cc:299] Negotiation successful
0812 11:10:12.512125 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 11:10:12.513418 30813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:12.498866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.193:43769)
0812 11:10:12.499720 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:12.499759 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:12.500583 (+ 824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:12.501278 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:12.501284 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:12.501649 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:12.502126 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:12.502138 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:12.503026 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:12.503071 (+ 45us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:12.503972 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:12.503983 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:12.504526 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:12.504531 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:12.504693 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:12.505252 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:12.505269 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:12.506998 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:12.509406 (+ 2408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:12.509415 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:12.509435 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:12.509862 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:12.510374 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:12.510377 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:12.510379 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:12.510569 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:12.511232 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:12.511243 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:12.511640 (+ 397us) client_negotiation.cc:769] Sending connection context
0812 11:10:12.511889 (+ 249us) client_negotiation.cc:240] Negotiation successful
0812 11:10:12.512277 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":211,"threads_started":1}
I20260812 11:10:12.515139 30810 heartbeater.cc:347] Connected to a master server at 127.17.44.254:42379
I20260812 11:10:12.515450 30810 heartbeater.cc:464] Registering TS with master...
I20260812 11:10:12.516090 30810 heartbeater.cc:511] Master 127.17.44.254:42379 requested a full tablet report, sending...
I20260812 11:10:12.518172 30511 ts_manager.cc:194] Registered new tserver with Master: 240e712ea353491986889786e0e3ce3b (127.17.44.193:40873)
WARNING: no policy specified for kudu/127.17.44.194@KRBTEST.COM; defaulting to no policy
I20260812 11:10:12.519804 30511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.193@KRBTEST.COM'} at 127.17.44.193:43769
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
I20260812 11:10:12.543280 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:42379
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:42669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:10:12.669392 30818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:12.669654 30818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:12.669697 30818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:12.673667 30818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:12.673733 30818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:12.673791 30818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:10:12.679276 30818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:42669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:42379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:12.680335 30818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:12.681371 30818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:12.691325 30828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:12.691514 30824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:12.691325 30826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:12.691618 30818 server_base.cc:1034] running on GCE node
I20260812 11:10:12.692186 30818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:12.692813 30818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:12.694036 30818 hybrid_clock.cc:648] HybridClock initialized: now 1786533012693998 us; error 43 us; skew 500 ppm
Aug 12 11:10:12 dist-test-slave-drl0 krb5kdc[30458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533012, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:10:12.696882 30818 init.cc:377] Logged in from keytab as kudu/127.17.44.194@KRBTEST.COM (short username kudu)
I20260812 11:10:12.697976 30818 webserver.cc:460] Webserver started at http://127.17.44.194:35121/ using document root <none> and password file <none>
I20260812 11:10:12.698640 30818 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:12.698709 30818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:12.699008 30818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:10:12.701268 30818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "45067899d81c4b39859dbd0bf7707862"
format_stamp: "Formatted at 2026-08-12 11:10:12 on dist-test-slave-drl0"
server_key: "3297e548204815da569206b9505dc28b"
server_key_iv: "3e7c7972f7e4c36f41a009f7dfdf75a4"
server_key_version: "encryptionkey@0"
I20260812 11:10:12.701889 30818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "45067899d81c4b39859dbd0bf7707862"
format_stamp: "Formatted at 2026-08-12 11:10:12 on dist-test-slave-drl0"
server_key: "3297e548204815da569206b9505dc28b"
server_key_iv: "3e7c7972f7e4c36f41a009f7dfdf75a4"
server_key_version: "encryptionkey@0"
I20260812 11:10:12.706342 30818 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 11:10:12.709441 30839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:12.711036 30818 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 11:10:12.711138 30818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "45067899d81c4b39859dbd0bf7707862"
format_stamp: "Formatted at 2026-08-12 11:10:12 on dist-test-slave-drl0"
server_key: "3297e548204815da569206b9505dc28b"
server_key_iv: "3e7c7972f7e4c36f41a009f7dfdf75a4"
server_key_version: "encryptionkey@0"
I20260812 11:10:12.711272 30818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:12.735283 30818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:12.738521 30818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:12.738729 30818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:12.739408 30818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:10:12.740224 30818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:10:12.740272 30818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:12.740319 30818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:10:12.740375 30818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:12.751320 30818 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:45811
I20260812 11:10:12.751379 31003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:45811 every 8 connection(s)
I20260812 11:10:12.752279 30818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:10:12.752532 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 30818
I20260812 11:10:12.752686 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:10:12.753074 17587 external_mini_cluster.cc:1428] Setting key 18bdcf620a623ff07cb82c937a77e8a1
Aug 12 11:10:12 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533012, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.44.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.195@KRBTEST.COM" created.
I20260812 11:10:12.769528 30575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:12.754208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:34053 (local address 127.17.44.254:42379)
0812 11:10:12.754356 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:12.754361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:12.755323 (+ 962us) server_negotiation.cc:407] Connection header received
0812 11:10:12.756621 (+ 1298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:12.756627 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:12.756689 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:12.756790 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:12.758506 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:12.759861 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:12.761183 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:12.761596 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:12.761731 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:12.764282 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:12.764287 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:12.764291 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:12.764322 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:12.766143 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:12.767315 (+ 1172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:12.767318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:12.767319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:12.767405 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:12.768102 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:12.768229 (+ 127us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:12.768233 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:12.768452 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:12.768534 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:12.769275 (+ 741us) server_negotiation.cc:299] Negotiation successful
0812 11:10:12.769405 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 11:10:12.770628 31009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:12.754665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.194:34053)
0812 11:10:12.755161 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:12.755212 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:12.756425 (+ 1213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:12.756990 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:12.757002 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:12.757504 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:12.758315 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:12.758342 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:12.760004 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:12.760007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:12.761062 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:12.761073 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:12.761770 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:12.761773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:12.761924 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:12.762562 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:12.762578 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:12.764102 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:12.766301 (+ 2199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:12.766306 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:12.766317 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:12.767188 (+ 871us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:12.767521 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:12.767523 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:12.767525 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:12.767667 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:12.768642 (+ 975us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:12.768648 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:12.768955 (+ 307us) client_negotiation.cc:769] Sending connection context
0812 11:10:12.769172 (+ 217us) client_negotiation.cc:240] Negotiation successful
0812 11:10:12.769517 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":139,"threads_started":1}
I20260812 11:10:12.772037 31004 heartbeater.cc:347] Connected to a master server at 127.17.44.254:42379
I20260812 11:10:12.772293 31004 heartbeater.cc:464] Registering TS with master...
I20260812 11:10:12.772781 31004 heartbeater.cc:511] Master 127.17.44.254:42379 requested a full tablet report, sending...
I20260812 11:10:12.774163 30511 ts_manager.cc:194] Registered new tserver with Master: 45067899d81c4b39859dbd0bf7707862 (127.17.44.194:45811)
I20260812 11:10:12.775277 30511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.194@KRBTEST.COM'} at 127.17.44.194:34053
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
I20260812 11:10:12.789628 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:42379
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:42669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:10:12.915846 31013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:12.916100 31013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:12.916169 31013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:12.920267 31013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:12.920356 31013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:12.920440 31013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:10:12.925683 31013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:42669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:42379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:12.926698 31013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:12.927806 31013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:12.938885 31022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:12.939441 31026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:12.939810 31021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:12.939888 31013 server_base.cc:1034] running on GCE node
I20260812 11:10:12.940375 31013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:12.941289 31013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:12.942469 31013 hybrid_clock.cc:648] HybridClock initialized: now 1786533012942445 us; error 39 us; skew 500 ppm
Aug 12 11:10:12 dist-test-slave-drl0 krb5kdc[30458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533012, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:10:12.949513 31013 init.cc:377] Logged in from keytab as kudu/127.17.44.195@KRBTEST.COM (short username kudu)
I20260812 11:10:12.950665 31013 webserver.cc:460] Webserver started at http://127.17.44.195:38761/ using document root <none> and password file <none>
I20260812 11:10:12.951339 31013 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:12.951431 31013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:12.951649 31013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:10:12.955050 31013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "150610d5dc0a40d2a5724266916aac64"
format_stamp: "Formatted at 2026-08-12 11:10:12 on dist-test-slave-drl0"
server_key: "21be4f63975ef53433b81053c0fa76b9"
server_key_iv: "34fe818ec3352c395f9feb4cf9238394"
server_key_version: "encryptionkey@0"
I20260812 11:10:12.956015 31013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "150610d5dc0a40d2a5724266916aac64"
format_stamp: "Formatted at 2026-08-12 11:10:12 on dist-test-slave-drl0"
server_key: "21be4f63975ef53433b81053c0fa76b9"
server_key_iv: "34fe818ec3352c395f9feb4cf9238394"
server_key_version: "encryptionkey@0"
I20260812 11:10:12.961531 31013 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 11:10:12.965641 31034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:12.967617 31013 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 11:10:12.967779 31013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "150610d5dc0a40d2a5724266916aac64"
format_stamp: "Formatted at 2026-08-12 11:10:12 on dist-test-slave-drl0"
server_key: "21be4f63975ef53433b81053c0fa76b9"
server_key_iv: "34fe818ec3352c395f9feb4cf9238394"
server_key_version: "encryptionkey@0"
I20260812 11:10:12.967936 31013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:12.982700 31013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:12.986271 31013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:12.986485 31013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:12.987161 31013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:10:12.988029 31013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:10:12.988102 31013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:12.988175 31013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:10:12.988220 31013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:13.000784 31013 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:35139
I20260812 11:10:13.000808 31219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:35139 every 8 connection(s)
I20260812 11:10:13.001861 31013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:10:13.008659 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 31013
I20260812 11:10:13.009002 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:10:13.009317 17587 external_mini_cluster.cc:1428] Setting key 0b946549bd74df1e19923a79ead05c93
Aug 12 11:10:13 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533012, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:10:13.017730 30575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:13.004084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:51453 (local address 127.17.44.254:42379)
0812 11:10:13.004312 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:13.004320 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:13.005381 (+ 1061us) server_negotiation.cc:407] Connection header received
0812 11:10:13.006714 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:13.006717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:13.006781 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:13.006894 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:13.008092 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.008638 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.010366 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.010666 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.010777 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:13.013502 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:13.013505 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:13.013507 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:13.013530 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:13.015170 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:13.015723 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:13.015725 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:13.015727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:13.015806 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:13.016643 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:13.016646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:13.016647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:13.016850 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:13.016941 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:13.017488 (+ 547us) server_negotiation.cc:299] Negotiation successful
0812 11:10:13.017594 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 11:10:13.018493 31223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:13.004307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.195:51453)
0812 11:10:13.005190 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:13.005255 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:13.006500 (+ 1245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:13.007093 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:13.007099 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:13.007472 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:13.007946 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:13.007959 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.008999 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.009009 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:13.010199 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:13.010217 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.010869 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.010873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:13.011084 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:13.011848 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:13.011865 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:13.013327 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:13.015281 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:13.015285 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:13.015296 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:13.015581 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:13.016133 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:13.016138 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:13.016139 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:13.016436 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:13.016992 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:13.016997 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:13.017241 (+ 244us) client_negotiation.cc:769] Sending connection context
0812 11:10:13.017410 (+ 169us) client_negotiation.cc:240] Negotiation successful
0812 11:10:13.017652 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":147,"threads_started":1}
I20260812 11:10:13.020074 31220 heartbeater.cc:347] Connected to a master server at 127.17.44.254:42379
I20260812 11:10:13.020331 31220 heartbeater.cc:464] Registering TS with master...
I20260812 11:10:13.020831 31220 heartbeater.cc:511] Master 127.17.44.254:42379 requested a full tablet report, sending...
I20260812 11:10:13.021952 30511 ts_manager.cc:194] Registered new tserver with Master: 150610d5dc0a40d2a5724266916aac64 (127.17.44.195:35139)
I20260812 11:10:13.022516 30511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.195@KRBTEST.COM'} at 127.17.44.195:51453
I20260812 11:10:13.023660 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.193.keytab.
Entry for principal oryx/127.17.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.193.keytab.
WARNING: no policy specified for oryx/127.17.44.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.44.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.194.keytab.
Entry for principal oryx/127.17.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.194.keytab.
WARNING: no policy specified for oryx/127.17.44.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.195.keytab.
Entry for principal oryx/127.17.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.195.keytab.
I20260812 11:10:13.119128 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 30616
I20260812 11:10:13.133044 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 30818
I20260812 11:10:13.146435 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 31013
I20260812 11:10:13.160013 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 30472
I20260812 11:10:13.172857 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:42379
--webserver_interface=127.17.44.254
--webserver_port=36493
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:42669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:42379
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:10:13.293690 31234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:13.293946 31234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:13.293987 31234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:13.297959 31234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:10:13.298008 31234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:13.298027 31234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:13.298044 31234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:10:13.298059 31234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:10:13.303375 31234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:42669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:42379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:42379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=36493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:13.304419 31234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:13.305857 31234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:13.314477 31245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:13.314491 31242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:13.314805 31234 server_base.cc:1034] running on GCE node
W20260812 11:10:13.314494 31251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:13.315424 31234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:13.316457 31234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:13.317675 31234 hybrid_clock.cc:648] HybridClock initialized: now 1786533013317647 us; error 40 us; skew 500 ppm
Aug 12 11:10:13 dist-test-slave-drl0 krb5kdc[30458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533013, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:10:13.321679 31234 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:10:13.322607 31234 webserver.cc:460] Webserver started at http://127.17.44.254:36493/ using document root <none> and password file <none>
I20260812 11:10:13.323374 31234 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:13.323523 31234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:13.327389 31234 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:10:13.329625 31263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:13.331377 31234 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:10:13.331532 31234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "81b078f8a5454db5abbc2804776ae1d9"
format_stamp: "Formatted at 2026-08-12 11:10:12 on dist-test-slave-drl0"
server_key: "87e9b2bb7e2fd051d27b36f234fe6f2d"
server_key_iv: "d81db2607eb7e4d28cf2d2442de7bd27"
server_key_version: "encryptionkey@0"
I20260812 11:10:13.332085 31234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:13.362066 31234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:13.365459 31234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:13.365682 31234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:13.373899 31234 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:42379
I20260812 11:10:13.373929 31342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:42379 every 8 connection(s)
I20260812 11:10:13.375070 31234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:10:13.379616 31344 sys_catalog.cc:263] Verifying existing consensus state
I20260812 11:10:13.381711 31344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9: Bootstrap starting.
I20260812 11:10:13.382055 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 31234
I20260812 11:10:13.382539 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:40873
--local_ip_for_outbound_sockets=127.17.44.193
--tserver_master_addrs=127.17.44.254:42379
--webserver_port=37289
--webserver_interface=127.17.44.193
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:42669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.44.193
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:10:13.401351 31344 log.cc:824] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9: Log is configured to *not* fsync() on all Append() calls
I20260812 11:10:13.408093 31344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 11:10:13.408475 31344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9: Bootstrap complete.
I20260812 11:10:13.413821 31344 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b078f8a5454db5abbc2804776ae1d9" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42379 } }
I20260812 11:10:13.414440 31344 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81b078f8a5454db5abbc2804776ae1d9, State: Initialized, Role: FOLLOWER
I20260812 11:10:13.415015 31344 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b078f8a5454db5abbc2804776ae1d9" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42379 } }
I20260812 11:10:13.415164 31344 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:10:13.415237 31344 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:10:13.415340 31344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [term 1 FOLLOWER]: Advancing to term 2
I20260812 11:10:13.417322 31344 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b078f8a5454db5abbc2804776ae1d9" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42379 } }
I20260812 11:10:13.417692 31344 leader_election.cc:302] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81b078f8a5454db5abbc2804776ae1d9; no voters:
I20260812 11:10:13.418051 31344 leader_election.cc:288] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 11:10:13.418159 31348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 11:10:13.418421 31348 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [term 2 LEADER]: Becoming Leader. State: Replica: 81b078f8a5454db5abbc2804776ae1d9, State: Running, Role: LEADER
I20260812 11:10:13.418730 31348 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b078f8a5454db5abbc2804776ae1d9" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42379 } }
I20260812 11:10:13.419287 31344 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:10:13.420322 31349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "81b078f8a5454db5abbc2804776ae1d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b078f8a5454db5abbc2804776ae1d9" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42379 } } }
I20260812 11:10:13.420454 31349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [sys.catalog]: This master's current role is: LEADER
I20260812 11:10:13.421366 31362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:10:13.421438 31350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81b078f8a5454db5abbc2804776ae1d9. Latest consensus state: current_term: 2 leader_uuid: "81b078f8a5454db5abbc2804776ae1d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b078f8a5454db5abbc2804776ae1d9" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42379 } } }
I20260812 11:10:13.421530 31350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9 [sys.catalog]: This master's current role is: LEADER
I20260812 11:10:13.424633 31362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:10:13.425843 31362 catalog_manager.cc:1259] Loaded cluster ID: 42218549de974fcf8d4d41fc1f3d89da
I20260812 11:10:13.425918 31362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:10:13.428596 31362 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:10:13.429281 31362 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 81b078f8a5454db5abbc2804776ae1d9: Loaded TSK: 0
I20260812 11:10:13.430850 31362 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 11:10:13.515883 31346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:13.516286 31346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:13.516382 31346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:13.521764 31346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:13.521879 31346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:13.521998 31346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:10:13.528554 31346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:42669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.193.keytab
--principal=oryx/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:40873
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=37289
--tserver_master_addrs=127.17.44.254:42379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:13.529639 31346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:13.530664 31346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:13.540771 31386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:13.540771 31385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:13.541049 31346 server_base.cc:1034] running on GCE node
W20260812 11:10:13.541270 31389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:13.541801 31346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:13.542718 31346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:13.543947 31346 hybrid_clock.cc:648] HybridClock initialized: now 1786533013543940 us; error 60 us; skew 500 ppm
Aug 12 11:10:13 dist-test-slave-drl0 krb5kdc[30458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533013, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:10:13.547451 31346 init.cc:377] Logged in from keytab as oryx/127.17.44.193@KRBTEST.COM (short username oryx)
I20260812 11:10:13.548381 31346 webserver.cc:460] Webserver started at http://127.17.44.193:37289/ using document root <none> and password file <none>
I20260812 11:10:13.549016 31346 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:13.549149 31346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:13.554458 31346 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 11:10:13.556910 31400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:13.558259 31346 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:10:13.558509 31346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "240e712ea353491986889786e0e3ce3b"
format_stamp: "Formatted at 2026-08-12 11:10:12 on dist-test-slave-drl0"
server_key: "8db9fb01c3ad368e834883ec7f12ec4f"
server_key_iv: "3b4099b260ce2a86a143f8a9c53ec8f8"
server_key_version: "encryptionkey@0"
I20260812 11:10:13.559162 31346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:13.572942 31346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:13.576282 31346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:13.576505 31346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:13.577116 31346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:10:13.578095 31346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:10:13.578197 31346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:13.578274 31346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:10:13.578325 31346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:13.588469 31346 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:40873
I20260812 11:10:13.588548 31570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:40873 every 8 connection(s)
I20260812 11:10:13.589486 31346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:10:13.591221 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 31346
I20260812 11:10:13.591784 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:45811
--local_ip_for_outbound_sockets=127.17.44.194
--tserver_master_addrs=127.17.44.254:42379
--webserver_port=35121
--webserver_interface=127.17.44.194
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:42669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.44.194
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 11:10:13 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:13 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 11:10:13.604306 31575 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:13.592010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:41453 (local address 127.17.44.254:42379)
0812 11:10:13.592488 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:13.592513 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:13.593357 (+ 844us) server_negotiation.cc:407] Connection header received
0812 11:10:13.594431 (+ 1074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:13.594441 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:13.594700 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:13.595033 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:13.596153 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.597110 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.598326 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.598768 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.599001 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:13.603581 (+ 4580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.193:41453: BlockingRecv error: recv got EOF from 127.17.44.193:41453 (error 108)
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":139,"threads_started":1}
I20260812 11:10:13.604332 31576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:13.592094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.193:41453)
0812 11:10:13.593172 (+ 1078us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:13.593229 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:13.594042 (+ 813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:13.595108 (+ 1066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:13.595116 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:13.595550 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:13.595980 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:13.595993 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.597252 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.597256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:13.598195 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:13.598206 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.598994 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.598999 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:13.599190 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:13.599890 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:13.599915 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:13.603382 (+ 3467us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":814,"thread_start_us":433,"threads_started":1}
W20260812 11:10:13.604964 31571 heartbeater.cc:644] Failed to heartbeat to 127.17.44.254:42379 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.44.254:42379: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Aug 12 11:10:13 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:13 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:13.611925 31576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:13.605437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.193:54239)
0812 11:10:13.605530 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:13.605546 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:13.605638 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:13.605974 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:13.605977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:13.606166 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:13.606261 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:13.606267 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.607186 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.607190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:13.607872 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:13.607882 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.608721 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.608726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:13.608852 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:13.609544 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:13.609558 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:13.611755 (+ 2197us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 11:10:13.612182 31575 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:13.605528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:54239 (local address 127.17.44.254:42379)
0812 11:10:13.605635 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:13.605640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:13.605654 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:10:13.605729 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:13.605731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:13.605773 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:13.605974 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:13.606410 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.607052 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.608005 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.608458 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.608703 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:13.611944 (+ 3241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.193:54239: BlockingRecv error: recv got EOF from 127.17.44.193:54239 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 12 11:10:13 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:13 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:13.618036 31576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:13.612643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.193:41925)
0812 11:10:13.612740 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:13.612757 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:13.612857 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:13.613101 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:13.613103 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:13.613274 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:13.613356 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:13.613362 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.614271 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.614282 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:13.614921 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:13.614982 (+ 61us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.615448 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.615451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:13.615530 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:13.616079 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:13.616091 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:13.617894 (+ 1803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 11:10:13.618186 31575 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:13.612736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:41925 (local address 127.17.44.254:42379)
0812 11:10:13.612848 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:13.612853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:13.612867 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:10:13.612948 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:13.612951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:13.612990 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:13.613085 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:13.613469 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.614125 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.615102 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.615333 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.615447 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:13.618072 (+ 2625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.193:41925: BlockingRecv error: recv got EOF from 127.17.44.193:41925 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260812 11:10:13.618494 31571 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 11:10:13.720875 31577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:13.721135 31577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:13.721177 31577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:13.725360 31577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:13.725431 31577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:13.725488 31577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:10:13.730645 31577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:42669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.194.keytab
--principal=oryx/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:45811
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=35121
--tserver_master_addrs=127.17.44.254:42379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:13.731942 31577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:13.733618 31577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:13.746608 31592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:13.746549 31596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:13.746625 31577 server_base.cc:1034] running on GCE node
W20260812 11:10:13.746555 31591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:13.747215 31577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:13.748229 31577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:13.749562 31577 hybrid_clock.cc:648] HybridClock initialized: now 1786533013749520 us; error 57 us; skew 500 ppm
Aug 12 11:10:13 dist-test-slave-drl0 krb5kdc[30458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533013, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:10:13.753412 31577 init.cc:377] Logged in from keytab as oryx/127.17.44.194@KRBTEST.COM (short username oryx)
I20260812 11:10:13.754370 31577 webserver.cc:460] Webserver started at http://127.17.44.194:35121/ using document root <none> and password file <none>
I20260812 11:10:13.755146 31577 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:13.755270 31577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:13.759413 31577 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260812 11:10:13.761807 31606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:13.763336 31577 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:10:13.763474 31577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "45067899d81c4b39859dbd0bf7707862"
format_stamp: "Formatted at 2026-08-12 11:10:12 on dist-test-slave-drl0"
server_key: "3297e548204815da569206b9505dc28b"
server_key_iv: "3e7c7972f7e4c36f41a009f7dfdf75a4"
server_key_version: "encryptionkey@0"
I20260812 11:10:13.764093 31577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:13.781284 31577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:13.785053 31577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:13.785274 31577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:13.786267 31577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:10:13.787583 31577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:10:13.787670 31577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:13.787748 31577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:10:13.787794 31577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:13.799324 31577 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:45811
I20260812 11:10:13.799394 31767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:45811 every 8 connection(s)
I20260812 11:10:13.800396 31577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:10:13.801461 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 31577
I20260812 11:10:13.801951 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:35139
--local_ip_for_outbound_sockets=127.17.44.195
--tserver_master_addrs=127.17.44.254:42379
--webserver_port=38761
--webserver_interface=127.17.44.195
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:42669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.44.195
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 11:10:13 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:13 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 11:10:13.812381 31575 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:13.802741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:52109 (local address 127.17.44.254:42379)
0812 11:10:13.802996 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:13.803006 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:13.803945 (+ 939us) server_negotiation.cc:407] Connection header received
0812 11:10:13.805333 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:13.805336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:13.805396 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:13.805474 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:13.806692 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.807409 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.808530 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.808759 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.808875 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:13.812251 (+ 3376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.194:52109: BlockingRecv error: recv got EOF from 127.17.44.194:52109 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 11:10:13.812973 31773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:13.802995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.194:52109)
0812 11:10:13.803746 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:13.803797 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:13.805080 (+ 1283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:13.805692 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:13.805702 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:13.806063 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:13.806532 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:13.806545 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.807539 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.807548 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:13.808399 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:13.808408 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.808916 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.808920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:13.809087 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:13.809698 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:13.809714 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:13.812122 (+ 2408us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":142,"threads_started":1}
W20260812 11:10:13.813423 31769 heartbeater.cc:644] Failed to heartbeat to 127.17.44.254:42379 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.44.254:42379: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Aug 12 11:10:13 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:13 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:13.820695 31773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:13.814159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.194:39015)
0812 11:10:13.814584 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:13.814606 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:13.814748 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:13.815133 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:13.815136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:13.815345 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:13.815602 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:13.815620 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.816459 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.816463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:13.817379 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:13.817391 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.817914 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.817918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:13.818013 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:13.818519 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:13.818533 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:13.820576 (+ 2043us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":284}
W20260812 11:10:13.820876 31575 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:13.814307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:39015 (local address 127.17.44.254:42379)
0812 11:10:13.814467 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:13.814494 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:13.814723 (+ 229us) server_negotiation.cc:407] Connection header received
0812 11:10:13.814867 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:13.814869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:13.814910 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:13.815065 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:13.815791 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.816333 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.817555 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.817779 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.817876 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:13.820732 (+ 2856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.194:39015: BlockingRecv error: recv got EOF from 127.17.44.194:39015 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 12 11:10:13 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:13 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:13.826988 31773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:13.821359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.194:41829)
0812 11:10:13.821481 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:13.821494 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:13.821560 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:13.821924 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:13.821927 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:13.822064 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:13.822139 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:13.822144 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.823261 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.823264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:13.823755 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:13.823763 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.824341 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.824345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:13.824413 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:13.824953 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:13.824967 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:13.826824 (+ 1857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 11:10:13.827442 31769 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 11:10:13.827853 31575 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:13.821589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:41829 (local address 127.17.44.254:42379)
0812 11:10:13.821690 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:13.821694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:13.821707 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:10:13.821743 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:13.821746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:13.821782 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:13.821878 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:13.822264 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.823140 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.823872 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:13.824209 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:13.824308 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:13.827721 (+ 3413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.194:41829: BlockingRecv error: recv got EOF from 127.17.44.194:41829 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260812 11:10:13.927449 31774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:13.927903 31774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:13.928018 31774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:13.932319 31774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:13.932416 31774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:13.932503 31774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:10:13.937793 31774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:42669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/krb5kdc/oryx_127.17.44.195.keytab
--principal=oryx/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:35139
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=38761
--tserver_master_addrs=127.17.44.254:42379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:13.938818 31774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:13.939929 31774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:13.949647 31781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:13.949654 31787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:13.950057 31774 server_base.cc:1034] running on GCE node
W20260812 11:10:13.950103 31782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:13.950521 31774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:13.951270 31774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:13.952474 31774 hybrid_clock.cc:648] HybridClock initialized: now 1786533013952432 us; error 65 us; skew 500 ppm
Aug 12 11:10:13 dist-test-slave-drl0 krb5kdc[30458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533013, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:10:13.955505 31774 init.cc:377] Logged in from keytab as oryx/127.17.44.195@KRBTEST.COM (short username oryx)
I20260812 11:10:13.956339 31774 webserver.cc:460] Webserver started at http://127.17.44.195:38761/ using document root <none> and password file <none>
I20260812 11:10:13.957015 31774 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:13.957103 31774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:13.961401 31774 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:10:13.963620 31795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:13.964931 31774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:10:13.965030 31774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "150610d5dc0a40d2a5724266916aac64"
format_stamp: "Formatted at 2026-08-12 11:10:12 on dist-test-slave-drl0"
server_key: "21be4f63975ef53433b81053c0fa76b9"
server_key_iv: "34fe818ec3352c395f9feb4cf9238394"
server_key_version: "encryptionkey@0"
I20260812 11:10:13.965619 31774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:13.987587 31774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:13.991117 31774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:13.991357 31774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:13.991998 31774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:10:13.992983 31774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:10:13.993060 31774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:13.993134 31774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:10:13.993182 31774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:14.004153 31774 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:35139
I20260812 11:10:14.004283 31973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:35139 every 8 connection(s)
I20260812 11:10:14.005165 31774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:10:14.010612 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 31774
Aug 12 11:10:14 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:14 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 11:10:14.017710 31575 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:14.007110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:35379 (local address 127.17.44.254:42379)
0812 11:10:14.007304 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:14.007308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:14.008374 (+ 1066us) server_negotiation.cc:407] Connection header received
0812 11:10:14.009280 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:14.009283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:14.009339 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:14.009448 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:14.010570 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:14.011387 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:14.013306 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:14.013647 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:14.013720 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:14.017580 (+ 3860us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.195:35379: BlockingRecv error: recv got EOF from 127.17.44.195:35379 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 11:10:14.018414 31977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:14.007495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.195:35379)
0812 11:10:14.008213 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:14.008261 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:14.009073 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:14.009630 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:14.009636 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:14.009976 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:14.010427 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:14.010439 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:14.011542 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:14.011546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:14.013168 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:14.013181 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:14.013819 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:14.013823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:14.014056 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:14.014687 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:14.014714 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:14.017490 (+ 2776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":119,"threads_started":1}
W20260812 11:10:14.019284 31975 heartbeater.cc:644] Failed to heartbeat to 127.17.44.254:42379 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.44.254:42379: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
I20260812 11:10:14.022291 31980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:14.011777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44790 (local address 127.17.44.254:42379)
0812 11:10:14.012024 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:14.012027 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:14.012037 (+ 10us) server_negotiation.cc:407] Connection header received
0812 11:10:14.012064 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:14.012066 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:14.012097 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:14.012200 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:14.012901 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:14.013493 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:14.014814 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:14.015090 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:14.015172 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:14.016950 (+ 1778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:14.016957 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:14.016976 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:14.017012 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:14.019359 (+ 2347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:14.019952 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:14.019958 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:14.019964 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:14.020110 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:14.020467 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:14.020471 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:14.020472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:14.020903 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:14.021105 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:14.021805 (+ 700us) server_negotiation.cc:299] Negotiation successful
0812 11:10:14.021996 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":104,"threads_started":1}
Aug 12 11:10:14 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:14 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:14.026647 31977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:14.019756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.195:43363)
0812 11:10:14.019920 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:14.019938 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:14.020005 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:14.020224 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:14.020226 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:14.020411 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:14.020598 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:14.020606 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:14.021597 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:14.021608 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:14.022687 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:14.022702 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:14.023445 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:14.023448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:14.023527 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:14.024070 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:14.024088 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:14.026554 (+ 2466us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100}
W20260812 11:10:14.026989 31575 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:14.019875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:43363 (local address 127.17.44.254:42379)
0812 11:10:14.019981 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:14.019985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:14.019997 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:10:14.020065 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:14.020067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:14.020104 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:14.020190 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:14.020697 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:14.021332 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:14.022871 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:14.023285 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:14.023412 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:14.026848 (+ 3436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.195:43363: BlockingRecv error: recv got EOF from 127.17.44.195:43363 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 11:10:14 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:14 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:14.032348 31977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:14.027313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.195:42099)
0812 11:10:14.027432 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:14.027447 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:14.027523 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:14.027790 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:14.027792 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:14.027912 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:14.027983 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:14.027987 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:14.028919 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:14.028922 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:14.029551 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:14.029562 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:14.030013 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:14.030016 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:14.030073 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:14.030540 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:14.030553 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:14.032267 (+ 1714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 11:10:14.032584 31975 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 11:10:14.032747 31575 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:14.027435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:42099 (local address 127.17.44.254:42379)
0812 11:10:14.027570 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:14.027574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:14.027585 (+ 11us) server_negotiation.cc:407] Connection header received
0812 11:10:14.027614 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:14.027616 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:14.027647 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:14.027717 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:14.028323 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:14.028793 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:14.029682 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:14.029907 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:14.029994 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:14.032619 (+ 2625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.195:42099: BlockingRecv error: recv got EOF from 127.17.44.195:42099 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 12 11:10:14 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:14 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:14.626022 31981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:14.619405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.193:58415)
0812 11:10:14.619688 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:14.619705 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:14.619815 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:14.620584 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:14.620651 (+ 67us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:14.620859 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:14.621008 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:14.621015 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:14.622058 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:14.622061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:14.622662 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:14.622671 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:14.623270 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:14.623273 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:14.623360 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:14.623962 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:14.623974 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:14.625846 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":120,"threads_started":1}
W20260812 11:10:14.626101 31982 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:14.619565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:58415 (local address 127.17.44.254:42379)
0812 11:10:14.619936 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:14.619941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:14.619958 (+ 17us) server_negotiation.cc:407] Connection header received
0812 11:10:14.620008 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:14.620013 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:14.620090 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:14.620404 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:14.621175 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:14.621901 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:14.622853 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:14.623129 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:14.623224 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:14.625975 (+ 2751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.193:58415: BlockingRecv error: recv got EOF from 127.17.44.193:58415 (error 108)
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":115,"threads_started":1}
Aug 12 11:10:14 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:14 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:14.834106 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:14.828458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.194:46443)
0812 11:10:14.828756 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:14.828773 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:14.828895 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:14.829173 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:14.829176 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:14.829396 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:14.829488 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:14.829494 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:14.830261 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:14.830268 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:14.830834 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:14.830842 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:14.831324 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:14.831327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:14.831411 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:14.831966 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:14.831978 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:14.833914 (+ 1936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":102,"threads_started":1}
W20260812 11:10:14.834139 31982 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:14.828661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:46443 (local address 127.17.44.254:42379)
0812 11:10:14.828814 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:14.828821 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:14.828839 (+ 18us) server_negotiation.cc:407] Connection header received
0812 11:10:14.829016 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:14.829019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:14.829064 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:14.829150 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:14.829604 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:14.830146 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:14.830978 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:14.831200 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:14.831319 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:14.834036 (+ 2717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.194:46443: BlockingRecv error: recv got EOF from 127.17.44.194:46443 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 11:10:15 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:15 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:15.040318 31985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:15.033418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.195:47927)
0812 11:10:15.033862 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:15.033877 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:15.033986 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:15.034755 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:15.034763 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:15.035203 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:15.035371 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:15.035382 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:15.036459 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:15.036461 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:15.037095 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:15.037105 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:15.037672 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:15.037676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:15.037742 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:15.038267 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:15.038279 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:15.040192 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":101,"threads_started":1}
W20260812 11:10:15.040521 31982 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:15.033498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:47927 (local address 127.17.44.254:42379)
0812 11:10:15.033867 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:15.033874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:15.033955 (+ 81us) server_negotiation.cc:407] Connection header received
0812 11:10:15.034071 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:15.034073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:15.034128 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:15.034203 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:15.035530 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:15.036333 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:15.037280 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:15.037526 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:15.037613 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:15.040379 (+ 2766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.195:47927: BlockingRecv error: recv got EOF from 127.17.44.195:47927 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 12 11:10:15 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:15 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:15.632881 31986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:15.626972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.193:56757)
0812 11:10:15.627262 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:15.627280 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:15.627401 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:15.627718 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:15.627721 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:15.628002 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:15.628127 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:15.628135 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:15.629085 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:15.629087 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:15.629629 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:15.629637 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:15.630127 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:15.630130 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:15.630196 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:15.630740 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:15.630752 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:15.632697 (+ 1945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":68,"threads_started":1}
W20260812 11:10:15.633638 31989 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:15.627052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:56757 (local address 127.17.44.254:42379)
0812 11:10:15.627365 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:15.627369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:15.627382 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:10:15.627523 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:15.627528 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:15.627584 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:15.627683 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:15.628314 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:15.628977 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:15.629785 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:15.630010 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:15.630095 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:15.633344 (+ 3249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.193:56757: BlockingRecv error: recv got EOF from 127.17.44.193:56757 (error 108)
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":138,"threads_started":1}
Aug 12 11:10:15 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:15 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:15.842166 31991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:15.835323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.194:33795)
0812 11:10:15.835800 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:15.835831 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:15.836001 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:15.836340 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:15.836343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:15.836548 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:15.836654 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:15.836659 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:15.837522 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:15.837524 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:15.838082 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:15.838090 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:15.838578 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:15.838581 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:15.838646 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:15.839459 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:15.839476 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:15.842021 (+ 2545us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":99,"threads_started":1}
W20260812 11:10:15.842406 31989 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:15.835573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:33795 (local address 127.17.44.254:42379)
0812 11:10:15.835794 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:15.835798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:15.835974 (+ 176us) server_negotiation.cc:407] Connection header received
0812 11:10:15.836116 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:15.836119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:15.836171 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:15.836257 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:15.836845 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:15.837397 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:15.838220 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:15.838442 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:15.838531 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:15.842280 (+ 3749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.194:33795: BlockingRecv error: recv got EOF from 127.17.44.194:33795 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 12 11:10:16 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:16 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:16.047608 31992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:16.041248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.195:60723)
0812 11:10:16.041515 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:16.041531 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:16.041640 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:16.041949 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:16.041951 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:16.042171 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:16.042278 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:16.042283 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:16.043132 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:16.043137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:16.043885 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:16.043914 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:16.044639 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:16.044643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:16.044730 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:16.045391 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:16.045405 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:16.047409 (+ 2004us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":85,"threads_started":1}
W20260812 11:10:16.047669 31989 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:16.041447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:60723 (local address 127.17.44.254:42379)
0812 11:10:16.041592 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:16.041598 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:16.041612 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:10:16.041762 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:16.041767 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:16.041831 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:16.041922 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:16.042424 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:16.043025 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:16.044212 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:16.044478 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:16.044627 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:16.047552 (+ 2925us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.195:60723: BlockingRecv error: recv got EOF from 127.17.44.195:60723 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 11:10:16 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:16 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:16.640568 31993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:16.634200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.193:42785)
0812 11:10:16.634587 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:16.634603 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:16.634709 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:16.635176 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:16.635179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:16.635373 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:16.635537 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:16.635544 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:16.636660 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:16.636663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:16.637327 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:16.637336 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:16.637866 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:16.637869 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:16.637946 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:16.638498 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:16.638510 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:16.640439 (+ 1929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":131,"threads_started":1}
W20260812 11:10:16.640792 31994 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:16.634337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:42785 (local address 127.17.44.254:42379)
0812 11:10:16.634671 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:16.634675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:16.634692 (+ 17us) server_negotiation.cc:407] Connection header received
0812 11:10:16.634953 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:16.634956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:16.635012 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:16.635101 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:16.635786 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:16.636483 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:16.637467 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:16.637714 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:16.637810 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:16.640615 (+ 2805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.193:42785: BlockingRecv error: recv got EOF from 127.17.44.193:42785 (error 108)
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":100,"threads_started":1}
Aug 12 11:10:16 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:16 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:16.849833 32006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:16.843269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.194:46907)
0812 11:10:16.843538 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:16.843553 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:16.843665 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:16.843938 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:16.843941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:16.844121 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:16.844215 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:16.844221 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:16.845174 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:16.845177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:16.845944 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:16.845953 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:16.846502 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:16.846508 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:16.846596 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:16.847260 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:16.847274 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:16.849717 (+ 2443us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":113,"threads_started":1}
W20260812 11:10:16.850086 31994 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:16.843357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:46907 (local address 127.17.44.254:42379)
0812 11:10:16.843525 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:16.843529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:16.843680 (+ 151us) server_negotiation.cc:407] Connection header received
0812 11:10:16.843757 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:16.843759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:16.843807 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:16.843874 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:16.844353 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:16.844959 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:16.846062 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:16.846379 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:16.846464 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:16.849931 (+ 3467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.194:46907: BlockingRecv error: recv got EOF from 127.17.44.194:46907 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 12 11:10:17 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:17 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:17.054553 32007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:17.048512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.195:33473)
0812 11:10:17.048808 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:17.048821 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:17.048923 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:17.049165 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:17.049168 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:17.049369 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:17.049465 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:17.049470 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:17.050296 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:17.050299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:17.050859 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:17.050867 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:17.051459 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:17.051462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:17.051529 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:17.052143 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:17.052156 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:17.054436 (+ 2280us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":117,"threads_started":1}
W20260812 11:10:17.054798 31994 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:17.048605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:33473 (local address 127.17.44.254:42379)
0812 11:10:17.048768 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:17.048772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:17.048937 (+ 165us) server_negotiation.cc:407] Connection header received
0812 11:10:17.048983 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:17.048986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:17.049034 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:17.049108 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:17.049627 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:17.050154 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:17.051088 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:17.051321 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:17.051412 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:17.054661 (+ 3249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.195:33473: BlockingRecv error: recv got EOF from 127.17.44.195:33473 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 12 11:10:17 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:17 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:17.651186 32011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:17.641404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.193:56763)
0812 11:10:17.641695 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:17.641715 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:17.641897 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:17.642193 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:17.642195 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:17.642492 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:17.642595 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:17.642601 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:17.643451 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:17.643454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:17.644112 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:17.644122 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:17.644674 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:17.644679 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:17.644783 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:17.649000 (+ 4217us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:17.649014 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:17.651061 (+ 2047us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":101,"threads_started":1}
W20260812 11:10:17.651352 32012 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:17.641519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:56763 (local address 127.17.44.254:42379)
0812 11:10:17.641859 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:17.641862 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:17.641879 (+ 17us) server_negotiation.cc:407] Connection header received
0812 11:10:17.641976 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:17.641979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:17.642025 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:17.642110 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:17.642743 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:17.643317 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:17.644211 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:17.644524 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:17.644644 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:17.651203 (+ 6559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.193:56763: BlockingRecv error: recv got EOF from 127.17.44.193:56763 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":90,"threads_started":1}
Aug 12 11:10:17 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:17 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:17.858053 32016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:17.850740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.194:55845)
0812 11:10:17.851054 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:17.851067 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:17.851169 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:17.851409 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:17.851411 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:17.851595 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:17.851688 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:17.851693 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:17.852862 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:17.852865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:17.853459 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:17.853468 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:17.854293 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:17.854298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:17.854394 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:17.855234 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:17.855249 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:17.857852 (+ 2603us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":113,"threads_started":1}
W20260812 11:10:17.858147 32012 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:17.850869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:55845 (local address 127.17.44.254:42379)
0812 11:10:17.851037 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:17.851040 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:17.851167 (+ 127us) server_negotiation.cc:407] Connection header received
0812 11:10:17.851255 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:17.851258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:17.851306 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:17.851372 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:17.851937 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:17.852684 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:17.853682 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:17.853976 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:17.854147 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:17.858039 (+ 3892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.194:55845: BlockingRecv error: recv got EOF from 127.17.44.194:55845 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 12 11:10:18 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:18 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:18.062361 32017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:18.055806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.195:57699)
0812 11:10:18.056154 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:18.056176 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:18.056324 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:18.056605 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:18.056609 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:18.056828 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:18.056924 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:18.056929 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:18.058172 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:18.058175 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:18.058737 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:18.058745 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:18.059325 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:18.059339 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:18.059426 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:18.060081 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:18.060092 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:18.062167 (+ 2075us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":102,"threads_started":1}
W20260812 11:10:18.062398 32012 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:18.055897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:57699 (local address 127.17.44.254:42379)
0812 11:10:18.056106 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:18.056111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:18.056256 (+ 145us) server_negotiation.cc:407] Connection header received
0812 11:10:18.056433 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:18.056437 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:18.056492 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:18.056585 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:18.057041 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:18.057699 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:18.058980 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:18.059203 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:18.059322 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:18.062296 (+ 2974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.195:57699: BlockingRecv error: recv got EOF from 127.17.44.195:57699 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 12 11:10:18 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:18 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:18.659583 32018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:18.652024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.193:34235)
0812 11:10:18.652335 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:18.652353 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:18.652482 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:18.652874 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:18.652876 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:18.653080 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:18.653175 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:18.653180 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:18.654443 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:18.654448 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:18.655125 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:18.655135 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:18.655913 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:18.655916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:18.655983 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:18.656678 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:18.656698 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:18.659359 (+ 2661us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":79,"threads_started":1}
W20260812 11:10:18.659787 32020 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:18.652190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:34235 (local address 127.17.44.254:42379)
0812 11:10:18.652590 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:18.652594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:18.652609 (+ 15us) server_negotiation.cc:407] Connection header received
0812 11:10:18.652653 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:18.652655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:18.652704 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:18.652789 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:18.653317 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:18.653959 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:18.655361 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:18.655739 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:18.655864 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:18.659592 (+ 3728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.193:34235: BlockingRecv error: recv got EOF from 127.17.44.193:34235 (error 108)
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":116,"threads_started":1}
Aug 12 11:10:18 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:18 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:18.865217 32022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:18.859090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.194:49071)
0812 11:10:18.859442 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:18.859460 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:18.859606 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:18.859883 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:18.859886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:18.860111 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:18.860204 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:18.860210 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:18.861003 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:18.861008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:18.861565 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:18.861573 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:18.862032 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:18.862036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:18.862121 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:18.862767 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:18.862783 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:18.865037 (+ 2254us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":111,"threads_started":1}
W20260812 11:10:18.865291 32020 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:18.859293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:49071 (local address 127.17.44.254:42379)
0812 11:10:18.859446 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:18.859452 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:18.859553 (+ 101us) server_negotiation.cc:407] Connection header received
0812 11:10:18.859712 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:18.859715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:18.859766 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:18.859857 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:18.860328 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:18.860867 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:18.861689 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:18.861914 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:18.862030 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:18.865159 (+ 3129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.194:49071: BlockingRecv error: recv got EOF from 127.17.44.194:49071 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 12 11:10:19 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:19 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:19.069591 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:19.063511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.195:51305)
0812 11:10:19.063805 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:19.063818 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:19.063921 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:19.064253 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:19.064256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:19.064455 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:19.064549 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:19.064554 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:19.065352 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:19.065354 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:19.065903 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:19.065921 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:19.066421 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:19.066424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:19.066494 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:19.067205 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:19.067218 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:19.069475 (+ 2257us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":114,"threads_started":1}
W20260812 11:10:19.069772 32020 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:19.063662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:51305 (local address 127.17.44.254:42379)
0812 11:10:19.063802 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:19.063806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:19.063895 (+ 89us) server_negotiation.cc:407] Connection header received
0812 11:10:19.063997 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:19.063999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:19.064045 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:19.064127 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:19.064691 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:19.065220 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:19.066047 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:19.066281 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:19.066370 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:19.069624 (+ 3254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.195:51305: BlockingRecv error: recv got EOF from 127.17.44.195:51305 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 12 11:10:19 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:19 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:19.667824 32025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:19.661080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.193:38785)
0812 11:10:19.661523 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:19.661628 (+ 105us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:19.661755 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:19.662042 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:19.662045 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:19.662238 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:19.662337 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:19.662342 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:19.663285 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:19.663288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:19.664051 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:19.664062 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:19.664737 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:19.664743 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:19.664897 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:19.665664 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:19.665676 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:19.667706 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":125,"threads_started":1}
W20260812 11:10:19.668061 32026 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:19.661310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:38785 (local address 127.17.44.254:42379)
0812 11:10:19.661573 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:19.661578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:19.661763 (+ 185us) server_negotiation.cc:407] Connection header received
0812 11:10:19.661844 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:19.661846 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:19.661896 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:19.661967 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:19.662509 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:19.663162 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:19.664200 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:19.664454 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:19.664568 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:19.667843 (+ 3275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.193:38785: BlockingRecv error: recv got EOF from 127.17.44.193:38785 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":98,"threads_started":1}
Aug 12 11:10:19 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:19 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:19.872268 32028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:19.866087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.194:37467)
0812 11:10:19.866442 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:19.866459 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:19.866609 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:19.866884 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:19.866887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:19.867153 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:19.867250 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:19.867256 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:19.868057 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:19.868063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:19.868764 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:19.868773 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:19.869323 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:19.869327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:19.869414 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:19.870010 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:19.870022 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:19.872091 (+ 2069us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":94,"threads_started":1}
W20260812 11:10:19.872325 32026 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:19.866240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:37467 (local address 127.17.44.254:42379)
0812 11:10:19.866427 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:19.866431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:19.866546 (+ 115us) server_negotiation.cc:407] Connection header received
0812 11:10:19.866709 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:19.866712 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:19.866767 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:19.866859 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:19.867406 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:19.867951 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:19.868893 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:19.869201 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:19.869310 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:19.872216 (+ 2906us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.194:37467: BlockingRecv error: recv got EOF from 127.17.44.194:37467 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 12 11:10:20 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:20 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:20.077688 32029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:20.070825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.195:48609)
0812 11:10:20.071235 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:20.071268 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:20.071388 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:20.071794 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:20.071798 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:20.072096 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:20.072250 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:20.072257 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:20.073289 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:20.073293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:20.073893 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:20.073902 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:20.074453 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:20.074457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:20.074639 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:20.075372 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:20.075399 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:20.077498 (+ 2099us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":159,"threads_started":1}
W20260812 11:10:20.077885 32026 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:20.071018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:48609 (local address 127.17.44.254:42379)
0812 11:10:20.071251 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:20.071256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:20.071389 (+ 133us) server_negotiation.cc:407] Connection header received
0812 11:10:20.071504 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:20.071507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:20.071598 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:20.071685 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:20.072424 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:20.073137 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:20.074052 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:20.074293 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:20.074390 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:20.077733 (+ 3343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.195:48609: BlockingRecv error: recv got EOF from 127.17.44.195:48609 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 12 11:10:20 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:20 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:20.675290 32031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:20.668751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.193:32781)
0812 11:10:20.669246 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:20.669280 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:20.669485 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:20.669952 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:20.669955 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:20.670142 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:20.670256 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:20.670261 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:20.671474 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:20.671478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:20.672121 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:20.672130 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:20.672612 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:20.672616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:20.672700 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:20.673281 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:20.673293 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:20.675117 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":103,"threads_started":1}
W20260812 11:10:20.675348 32032 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:20.668924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:32781 (local address 127.17.44.254:42379)
0812 11:10:20.669181 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:20.669185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:20.669405 (+ 220us) server_negotiation.cc:407] Connection header received
0812 11:10:20.669705 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:20.669708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:20.669778 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:20.669876 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:20.670426 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:20.671352 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:20.672246 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:20.672493 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:20.672606 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:20.675229 (+ 2623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.193:32781: BlockingRecv error: recv got EOF from 127.17.44.193:32781 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":102,"threads_started":1}
Aug 12 11:10:20 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:20 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:20.879935 32034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:20.873372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.194:53907)
0812 11:10:20.873747 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:20.873763 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:20.873900 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:20.874324 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:20.874327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:20.874571 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:20.874668 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:20.874674 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:20.875588 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:20.875590 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:20.876130 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:20.876138 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:20.876638 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:20.876641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:20.876721 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:20.877324 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:20.877347 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:20.879783 (+ 2436us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":97,"threads_started":1}
W20260812 11:10:20.880072 32032 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:20.873621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:53907 (local address 127.17.44.254:42379)
0812 11:10:20.873795 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:20.873801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:20.873815 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:10:20.874077 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:20.874080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:20.874138 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:20.874245 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:20.874807 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:20.875388 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:20.876274 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:20.876493 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:20.876594 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:20.879927 (+ 3333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.194:53907: BlockingRecv error: recv got EOF from 127.17.44.194:53907 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 12 11:10:21 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:21 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 11:10:21.085682 32032 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:21.078825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:51919 (local address 127.17.44.254:42379)
0812 11:10:21.079032 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:21.079036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:21.079629 (+ 593us) server_negotiation.cc:407] Connection header received
0812 11:10:21.079700 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:21.079703 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:21.079767 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:21.079864 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:21.080476 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:21.081033 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:21.081932 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:21.082150 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:21.082238 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:21.085586 (+ 3348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.195:51919: BlockingRecv error: recv got EOF from 127.17.44.195:51919 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 11:10:21.085789 32035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:21.079015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.195:51919)
0812 11:10:21.079499 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:21.079513 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:21.079624 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:21.079925 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:21.079928 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:21.080172 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:21.080333 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:21.080343 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:21.081218 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:21.081221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:21.081786 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:21.081794 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:21.082358 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:21.082362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:21.082467 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:21.083529 (+ 1062us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:21.083541 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:21.085429 (+ 1888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":154,"threads_started":1}
Aug 12 11:10:21 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:21 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:21.683662 32036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:21.676086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.193:40403)
0812 11:10:21.676486 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:21.676563 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:21.676692 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:21.677188 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:21.677190 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:21.677593 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:21.677943 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:21.677961 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:21.679486 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:21.679490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:21.680092 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:21.680100 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:21.680774 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:21.680777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:21.680847 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:21.681416 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:21.681428 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:21.683543 (+ 2115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":119,"threads_started":1}
W20260812 11:10:21.683892 32038 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:21.676303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:40403 (local address 127.17.44.254:42379)
0812 11:10:21.676566 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:21.676570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:21.676684 (+ 114us) server_negotiation.cc:407] Connection header received
0812 11:10:21.676766 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:21.676768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:21.676900 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:21.677135 (+ 235us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:21.678324 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:21.679313 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:21.680249 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:21.680513 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:21.680605 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:21.683735 (+ 3130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.193:40403: BlockingRecv error: recv got EOF from 127.17.44.193:40403 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":114,"threads_started":1}
Aug 12 11:10:21 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:21 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:21.886842 32039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:21.880789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.194:33589)
0812 11:10:21.881099 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:21.881112 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:21.881214 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:21.881551 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:21.881554 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:21.881738 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:21.881835 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:21.881840 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:21.882802 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:21.882805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:21.883495 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:21.883505 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:21.884199 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:21.884202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:21.884303 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:21.884890 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:21.884902 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:21.886666 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":132,"threads_started":1}
W20260812 11:10:21.886885 32038 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:21.880862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:33589 (local address 127.17.44.254:42379)
0812 11:10:21.881039 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:21.881043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:21.881187 (+ 144us) server_negotiation.cc:407] Connection header received
0812 11:10:21.881335 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:21.881337 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:21.881391 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:21.881466 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:21.881976 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:21.882654 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:21.883787 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:21.884064 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:21.884195 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:21.886785 (+ 2590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.194:33589: BlockingRecv error: recv got EOF from 127.17.44.194:33589 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 12 11:10:22 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:22 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:22.092944 32040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:22.086817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.195:59645)
0812 11:10:22.087284 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:22.087300 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:22.087467 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:22.087757 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:22.087760 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:22.087989 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:22.088085 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:22.088091 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:22.088864 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:22.088869 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:22.089419 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:22.089427 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:22.089891 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:22.089895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:22.089981 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:22.090565 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:22.090577 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:22.092663 (+ 2086us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":124,"threads_started":1}
W20260812 11:10:22.093210 32038 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:22.087017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:59645 (local address 127.17.44.254:42379)
0812 11:10:22.087284 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:22.087290 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:22.087398 (+ 108us) server_negotiation.cc:407] Connection header received
0812 11:10:22.087579 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:22.087582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:22.087639 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:22.087732 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:22.088210 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:22.088742 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:22.089545 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:22.089762 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:22.089876 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:22.093031 (+ 3155us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.195:59645: BlockingRecv error: recv got EOF from 127.17.44.195:59645 (error 108)
Metrics: {"server-negotiator.queue_time_us":163}
Aug 12 11:10:22 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:22 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:22.695336 32041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:22.684478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.193:44391)
0812 11:10:22.684786 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:22.684799 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:22.684898 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:22.685148 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:22.685150 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:22.685335 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:22.685440 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:22.685445 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:22.686249 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:22.686252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:22.686815 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:22.686824 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:22.687313 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:22.687316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:22.687383 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:22.692751 (+ 5368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:22.692770 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:22.695145 (+ 2375us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":97,"threads_started":1}
W20260812 11:10:22.695412 32043 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:22.684626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:44391 (local address 127.17.44.254:42379)
0812 11:10:22.684873 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:22.684876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:22.684890 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:10:22.684991 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:22.684994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:22.685040 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:22.685117 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:22.685573 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:22.686120 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:22.686964 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:22.687185 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:22.687272 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:22.695268 (+ 7996us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.193:44391: BlockingRecv error: recv got EOF from 127.17.44.193:44391 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":72,"threads_started":1}
Aug 12 11:10:22 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:22 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:22.893996 32045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:22.887802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.194:33563)
0812 11:10:22.888096 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:22.888116 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:22.888247 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:22.888531 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:22.888534 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:22.888757 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:22.888850 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:22.888856 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:22.889626 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:22.889631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:22.890179 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:22.890187 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:22.890646 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:22.890649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:22.890733 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:22.891287 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:22.891299 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:22.893719 (+ 2420us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":114,"threads_started":1}
W20260812 11:10:22.894076 32043 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:22.887861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:33563 (local address 127.17.44.254:42379)
0812 11:10:22.888045 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:22.888050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:22.888195 (+ 145us) server_negotiation.cc:407] Connection header received
0812 11:10:22.888357 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:22.888360 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:22.888414 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:22.888509 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:22.888980 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:22.889507 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:22.890308 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:22.890528 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:22.890643 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:22.893936 (+ 3293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.194:33563: BlockingRecv error: recv got EOF from 127.17.44.194:33563 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 12 11:10:23 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:23 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:23.102167 32046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:23.094083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.195:47575)
0812 11:10:23.094408 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:23.094422 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:23.094563 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:23.095224 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:23.095227 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:23.095548 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:23.095650 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:23.095656 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:23.096744 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:23.096750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:23.097310 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:23.097319 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:23.098034 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:23.098038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:23.098127 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:23.098753 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:23.098766 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:23.101838 (+ 3072us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":107,"threads_started":1}
W20260812 11:10:23.102289 32043 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:23.094391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:47575 (local address 127.17.44.254:42379)
0812 11:10:23.094541 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:23.094546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:23.094561 (+ 15us) server_negotiation.cc:407] Connection header received
0812 11:10:23.094674 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:23.094678 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:23.094817 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:23.095049 (+ 232us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:23.095796 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:23.096608 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:23.097440 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:23.097889 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:23.098033 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:23.102113 (+ 4080us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.195:47575: BlockingRecv error: recv got EOF from 127.17.44.195:47575 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 12 11:10:23 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:23 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:23.702643 32047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:23.696335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.193:36327)
0812 11:10:23.696651 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:23.696665 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:23.696756 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:23.697183 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:23.697187 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:23.697467 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:23.697634 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:23.697642 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:23.698596 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:23.698599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:23.699402 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:23.699413 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:23.699899 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:23.699903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:23.699988 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:23.700574 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:23.700586 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:23.702468 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":90,"threads_started":1}
W20260812 11:10:23.702708 32048 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:23.696400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:36327 (local address 127.17.44.254:42379)
0812 11:10:23.696732 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:23.696736 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:23.696752 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:10:23.696830 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:23.696833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:23.696878 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:23.696978 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:23.697846 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:23.698457 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:23.699546 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:23.699769 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:23.699892 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:23.702584 (+ 2692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.193:36327: BlockingRecv error: recv got EOF from 127.17.44.193:36327 (error 108)
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":105,"threads_started":1}
Aug 12 11:10:23 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:23 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:23.901502 32049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:23.895324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.194:49045)
0812 11:10:23.895639 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:23.895655 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:23.895774 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:23.896054 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:23.896056 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:23.896270 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:23.896367 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:23.896373 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:23.897164 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:23.897170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:23.897727 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:23.897735 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:23.898212 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:23.898215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:23.898295 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:23.899026 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:23.899038 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:23.901252 (+ 2214us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":87,"threads_started":1}
W20260812 11:10:23.901594 32048 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:23.895470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:49045 (local address 127.17.44.254:42379)
0812 11:10:23.895645 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:23.895651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:23.895716 (+ 65us) server_negotiation.cc:407] Connection header received
0812 11:10:23.895884 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:23.895888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:23.895941 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:23.896030 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:23.896491 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:23.897049 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:23.897868 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:23.898090 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:23.898197 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:23.901455 (+ 3258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.194:49045: BlockingRecv error: recv got EOF from 127.17.44.194:49045 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 12 11:10:24 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:24 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:24.109633 32051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:24.103240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.195:39063)
0812 11:10:24.103698 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:24.103715 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:24.103836 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:24.104096 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:24.104099 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:24.104336 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:24.104443 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:24.104448 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:24.105206 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:24.105211 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:24.105773 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:24.105781 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:24.106246 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:24.106249 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:24.106329 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:24.106896 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:24.106913 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:24.109461 (+ 2548us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":101,"threads_started":1}
W20260812 11:10:24.110205 32048 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:24.103392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:39063 (local address 127.17.44.254:42379)
0812 11:10:24.103553 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:24.103559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:24.103801 (+ 242us) server_negotiation.cc:407] Connection header received
0812 11:10:24.103938 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:24.103941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:24.103988 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:24.104077 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:24.104568 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:24.105093 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:24.105906 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:24.106125 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:24.106240 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:24.109650 (+ 3410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.195:39063: BlockingRecv error: recv got EOF from 127.17.44.195:39063 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 11:10:24 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:24 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:24.710758 32052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:24.703922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.193:33319)
0812 11:10:24.704235 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:24.704249 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:24.704359 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:24.704682 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:24.704684 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:24.704875 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:24.705070 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:24.705078 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:24.705896 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:24.705899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:24.706627 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:24.706638 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:24.707352 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:24.707355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:24.707444 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:24.708187 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:24.708206 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:24.710576 (+ 2370us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":97,"threads_started":1}
W20260812 11:10:24.710860 32053 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:24.703934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:33319 (local address 127.17.44.254:42379)
0812 11:10:24.704231 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:24.704235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:24.704370 (+ 135us) server_negotiation.cc:407] Connection header received
0812 11:10:24.704438 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:24.704441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:24.704490 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:24.704582 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:24.705223 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:24.705763 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:24.706777 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:24.707166 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:24.707346 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:24.710722 (+ 3376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.193:33319: BlockingRecv error: recv got EOF from 127.17.44.193:33319 (error 108)
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":97,"threads_started":1}
Aug 12 11:10:24 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:24 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:24.909292 32055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:24.902455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.194:43575)
0812 11:10:24.902890 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:24.902912 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:24.903164 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:24.903461 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:24.903463 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:24.903651 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:24.903749 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:24.903754 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:24.904579 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:24.904582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:24.905165 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:24.905173 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:24.905727 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:24.905731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:24.905837 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:24.906874 (+ 1037us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:24.906891 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:24.909140 (+ 2249us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":111,"threads_started":1}
W20260812 11:10:24.909503 32053 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:24.902578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:43575 (local address 127.17.44.254:42379)
0812 11:10:24.902796 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:24.902801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:24.903154 (+ 353us) server_negotiation.cc:407] Connection header received
0812 11:10:24.903256 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:24.903258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:24.903315 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:24.903380 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:24.903900 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:24.904448 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:24.905301 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:24.905530 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:24.905620 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:24.909323 (+ 3703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.194:43575: BlockingRecv error: recv got EOF from 127.17.44.194:43575 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 12 11:10:25 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:25 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:25.118290 32056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:25.110648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.195:33909)
0812 11:10:25.111070 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:25.111097 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:25.111245 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:25.111637 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:25.111641 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:25.111873 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:25.111989 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:25.112000 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:25.113386 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:25.113389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:25.114136 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:25.114148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:25.115240 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:25.115246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:25.115413 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:25.116074 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:25.116089 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:25.118154 (+ 2065us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":141,"threads_started":1}
W20260812 11:10:25.118530 32053 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:25.110822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:33909 (local address 127.17.44.254:42379)
0812 11:10:25.111043 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:25.111048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:25.111190 (+ 142us) server_negotiation.cc:407] Connection header received
0812 11:10:25.111358 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:25.111362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:25.111417 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:25.111624 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:25.112154 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:25.113040 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:25.114446 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:25.114899 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:25.115187 (+ 288us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:25.118403 (+ 3216us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.195:33909: BlockingRecv error: recv got EOF from 127.17.44.195:33909 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 12 11:10:25 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:25 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:25.719188 32057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:25.712036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.193:46631)
0812 11:10:25.712805 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:25.712824 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:25.712953 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:25.713478 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:25.713481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:25.713796 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:25.713968 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:25.713976 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:25.714923 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:25.714959 (+ 36us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:25.715548 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:25.715557 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:25.716158 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:25.716163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:25.716256 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:25.716922 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:25.716935 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:25.719029 (+ 2094us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":103,"threads_started":1}
W20260812 11:10:25.719316 32058 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:25.712338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:46631 (local address 127.17.44.254:42379)
0812 11:10:25.712963 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:25.712970 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:25.712989 (+ 19us) server_negotiation.cc:407] Connection header received
0812 11:10:25.713210 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:25.713213 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:25.713271 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:25.713404 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:25.714152 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:25.714768 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:25.715699 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:25.715979 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:25.716080 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:25.719183 (+ 3103us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.193:46631: BlockingRecv error: recv got EOF from 127.17.44.193:46631 (error 108)
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":90,"threads_started":1}
Aug 12 11:10:25 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:25 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:25.916524 32060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:25.910154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.194:47987)
0812 11:10:25.910420 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:25.910433 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:25.910536 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:25.910846 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:25.910848 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:25.911057 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:25.911159 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:25.911164 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:25.911985 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:25.911987 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:25.912539 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:25.912547 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:25.913016 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:25.913019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:25.913084 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:25.914076 (+ 992us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:25.914088 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:25.916396 (+ 2308us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":100,"threads_started":1}
W20260812 11:10:25.916687 32058 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:25.910222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:47987 (local address 127.17.44.254:42379)
0812 11:10:25.910387 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:25.910393 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:25.910507 (+ 114us) server_negotiation.cc:407] Connection header received
0812 11:10:25.910657 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:25.910660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:25.910712 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:25.910788 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:25.911352 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:25.911890 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:25.912664 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:25.912887 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:25.912973 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:25.916554 (+ 3581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.194:47987: BlockingRecv error: recv got EOF from 127.17.44.194:47987 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 12 11:10:26 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:26 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:26.125552 32061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:26.119269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.195:51981)
0812 11:10:26.119646 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:26.119666 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:26.119793 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:26.120248 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:26.120251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:26.120524 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:26.120705 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:26.120714 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:26.121737 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:26.121740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:26.122300 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:26.122308 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:26.122838 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:26.122841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:26.122907 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:26.123510 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:26.123522 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:26.125431 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":109,"threads_started":1}
W20260812 11:10:26.125808 32058 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:26.119678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:51981 (local address 127.17.44.254:42379)
0812 11:10:26.119802 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:26.119807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:26.119848 (+ 41us) server_negotiation.cc:407] Connection header received
0812 11:10:26.119897 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:26.119900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:26.119971 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:26.120083 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:26.120862 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:26.121604 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:26.122440 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:26.122669 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:26.122773 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:26.125690 (+ 2917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.195:51981: BlockingRecv error: recv got EOF from 127.17.44.195:51981 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 12 11:10:26 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:26 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:26.727506 32063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:26.720409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.193:34511)
0812 11:10:26.720729 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:26.720787 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:26.720931 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:26.721353 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:26.721356 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:26.721551 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:26.721687 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:26.721693 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:26.722662 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:26.722664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:26.723332 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:26.723341 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:26.723914 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:26.723916 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:26.723982 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:26.724773 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:26.724790 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:26.727372 (+ 2582us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":121,"threads_started":1}
W20260812 11:10:26.727622 32064 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:26.720541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:34511 (local address 127.17.44.254:42379)
0812 11:10:26.720890 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:26.720895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:26.720913 (+ 18us) server_negotiation.cc:407] Connection header received
0812 11:10:26.721075 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:26.721079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:26.721140 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:26.721313 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:26.721856 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:26.722504 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:26.723480 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:26.723749 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:26.723864 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:26.727493 (+ 3629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.193:34511: BlockingRecv error: recv got EOF from 127.17.44.193:34511 (error 108)
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":123,"threads_started":1}
Aug 12 11:10:26 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:26 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:26.924748 32065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:26.917553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.194:40179)
0812 11:10:26.917859 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:26.917874 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:26.917983 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:26.918413 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:26.918418 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:26.918749 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:26.918992 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:26.919002 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:26.919905 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:26.919908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:26.920737 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:26.920748 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:26.921325 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:26.921330 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:26.921479 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:26.922128 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:26.922143 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:26.924616 (+ 2473us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":123,"threads_started":1}
W20260812 11:10:26.924944 32064 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:26.917697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:40179 (local address 127.17.44.254:42379)
0812 11:10:26.917872 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:26.917876 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:26.917952 (+ 76us) server_negotiation.cc:407] Connection header received
0812 11:10:26.918063 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:26.918065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:26.918123 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:26.918204 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:26.919183 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:26.919755 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:26.920902 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:26.921192 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:26.921294 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:26.924780 (+ 3486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.194:40179: BlockingRecv error: recv got EOF from 127.17.44.194:40179 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 12 11:10:27 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:27 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:27.132741 32067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:27.126490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.195:40209)
0812 11:10:27.126735 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:27.126749 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:27.126863 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:27.127180 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:27.127183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:27.127373 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:27.127518 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:27.127526 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:27.128455 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:27.128458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:27.129021 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:27.129029 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:27.129755 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:27.129758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:27.129829 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:27.130404 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:27.130416 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:27.132586 (+ 2170us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":84,"threads_started":1}
W20260812 11:10:27.132824 32064 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:27.126506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:40209 (local address 127.17.44.254:42379)
0812 11:10:27.126637 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:27.126643 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:27.126824 (+ 181us) server_negotiation.cc:407] Connection header received
0812 11:10:27.126964 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:27.126967 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:27.127026 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:27.127105 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:27.127671 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:27.128309 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:27.129236 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:27.129544 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:27.129711 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:27.132729 (+ 3018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.195:40209: BlockingRecv error: recv got EOF from 127.17.44.195:40209 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 12 11:10:27 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:27 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:27.739408 32068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:27.728435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.193:47303)
0812 11:10:27.728856 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:27.728872 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:27.728967 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:27.729250 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:27.729252 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:27.729461 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:27.729562 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:27.729568 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:27.730454 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:27.730457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:27.731209 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:27.731219 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:27.731784 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:27.731788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:27.731859 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:27.737406 (+ 5547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:27.737423 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:27.739267 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":96,"threads_started":1}
W20260812 11:10:27.739638 32069 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:27.728662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:47303 (local address 127.17.44.254:42379)
0812 11:10:27.728989 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:27.728992 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:27.729019 (+ 27us) server_negotiation.cc:407] Connection header received
0812 11:10:27.729055 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:27.729057 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:27.729098 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:27.729169 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:27.729775 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:27.730329 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:27.731322 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:27.731650 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:27.731771 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:27.739488 (+ 7717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.193:47303: BlockingRecv error: recv got EOF from 127.17.44.193:47303 (error 108)
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":95,"threads_started":1}
Aug 12 11:10:27 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:27 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:27.932049 32071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:27.925705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.194:55301)
0812 11:10:27.926009 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:27.926022 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:27.926129 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:27.926388 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:27.926390 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:27.926570 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:27.926667 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:27.926671 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:27.927807 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:27.927813 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:27.928574 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:27.928585 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:27.929113 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:27.929116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:27.929179 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:27.929694 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:27.929706 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:27.931916 (+ 2210us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":112,"threads_started":1}
W20260812 11:10:27.932505 32069 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:27.925812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:55301 (local address 127.17.44.254:42379)
0812 11:10:27.925978 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:27.925983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:27.926156 (+ 173us) server_negotiation.cc:407] Connection header received
0812 11:10:27.926215 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:27.926218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:27.926268 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:27.926347 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:27.926817 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:27.927527 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:27.928735 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:27.928968 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:27.929055 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:27.932124 (+ 3069us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.194:55301: BlockingRecv error: recv got EOF from 127.17.44.194:55301 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 12 11:10:28 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:28 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.195@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:28.140343 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:28.133732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.195:47275)
0812 11:10:28.134014 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:28.134033 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:28.134134 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:28.134704 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:28.134707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:28.135335 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:28.135462 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:28.135473 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:28.136318 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:28.136321 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:28.136934 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:28.136942 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:28.137418 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:28.137422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:28.137504 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:28.138038 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:28.138050 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:28.140209 (+ 2159us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":109,"threads_started":1}
W20260812 11:10:28.140424 32069 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:28.134070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:47275 (local address 127.17.44.254:42379)
0812 11:10:28.134274 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:28.134279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:28.134298 (+ 19us) server_negotiation.cc:407] Connection header received
0812 11:10:28.134359 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:28.134363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:28.134430 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:28.134639 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:28.135610 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:28.136172 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:28.137071 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:28.137288 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:28.137412 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:28.140317 (+ 2905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.195:47275: BlockingRecv error: recv got EOF from 127.17.44.195:47275 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 12 11:10:28 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:28 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.193@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:28.747032 32074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:28.740693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.193:44535)
0812 11:10:28.741005 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:28.741020 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:28.741248 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:28.741677 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:28.741682 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:28.741891 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:28.742025 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:28.742032 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:28.743009 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:28.743013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:28.743570 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:28.743579 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:28.744096 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:28.744099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:28.744165 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:28.744813 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:28.744825 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:28.746874 (+ 2049us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":100,"threads_started":1}
W20260812 11:10:28.747277 32075 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:28.740719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:44535 (local address 127.17.44.254:42379)
0812 11:10:28.740999 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:28.741003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:28.741215 (+ 212us) server_negotiation.cc:407] Connection header received
0812 11:10:28.741350 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:28.741353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:28.741419 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:28.741522 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:28.742198 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:28.742777 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:28.743716 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:28.743959 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:28.744049 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:28.747135 (+ 3086us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.193:44535: BlockingRecv error: recv got EOF from 127.17.44.193:44535 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":100,"threads_started":1}
Aug 12 11:10:28 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 11:10:28 dist-test-slave-drl0 krb5kdc[30458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.44.194@KRBTEST.COM for oryx/127.17.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 11:10:28.941251 32076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:28.933058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42379 (local address 127.17.44.194:57949)
0812 11:10:28.933437 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:28.933450 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:28.933589 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:28.934313 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:28.934316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:28.934565 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:28.934722 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:28.934730 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:28.935881 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:28.935884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:28.936797 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:28.936825 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:28.937710 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:28.937712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:28.937782 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:28.938412 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:28.938425 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:28.941050 (+ 2625us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.44.254:42379: Server oryx/127.17.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":82,"threads_started":1}
W20260812 11:10:28.941313 32075 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:28.933134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:57949 (local address 127.17.44.254:42379)
0812 11:10:28.933296 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:28.933300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:28.933558 (+ 258us) server_negotiation.cc:407] Connection header received
0812 11:10:28.933763 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:28.933766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:28.934002 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:28.934146 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:28.934905 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:28.935741 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:28.937048 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:28.937416 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:28.937492 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:28.941196 (+ 3704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.44.194:57949: BlockingRecv error: recv got EOF from 127.17.44.194:57949 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 11:10:29.017643 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 31346
I20260812 11:10:29.032776 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 31577
I20260812 11:10:29.046916 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 31774
I20260812 11:10:29.058527 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 31234
2026-08-12T11:10:29Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18538 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T11:10:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:10:29Z Disabled control of system clock
I20260812 11:10:29.103180 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:46383
--webserver_interface=127.17.44.254
--webserver_port=0
--builtin_ntp_servers=127.17.44.212:41013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:46383
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 11:10:29.230445 32082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:29.230698 32082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:29.230732 32082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:29.234607 32082 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:10:29.234660 32082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:29.234675 32082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:29.234692 32082 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:10:29.234706 32082 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:10:29.239298 32082 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:41013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:46383
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:46383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:29.240787 32082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:29.242704 32082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:29.252570 32088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:29.252749 32082 server_base.cc:1034] running on GCE node
W20260812 11:10:29.252628 32089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:29.252601 32093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:29.253703 32082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:29.255231 32082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:29.256439 32082 hybrid_clock.cc:648] HybridClock initialized: now 1786533029256405 us; error 48 us; skew 500 ppm
I20260812 11:10:29.258594 32082 webserver.cc:460] Webserver started at http://127.17.44.254:39379/ using document root <none> and password file <none>
I20260812 11:10:29.259663 32082 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:29.259766 32082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:29.260249 32082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:10:29.262781 32082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "8eee5fdbfe18417999652c900f3bee36"
format_stamp: "Formatted at 2026-08-12 11:10:29 on dist-test-slave-drl0"
server_key: "8e809239e8e9954c80096557967a863e"
server_key_iv: "4d8ad0d0d362ffde6d2567be0a43ee7e"
server_key_version: "encryptionkey@0"
I20260812 11:10:29.263401 32082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "8eee5fdbfe18417999652c900f3bee36"
format_stamp: "Formatted at 2026-08-12 11:10:29 on dist-test-slave-drl0"
server_key: "8e809239e8e9954c80096557967a863e"
server_key_iv: "4d8ad0d0d362ffde6d2567be0a43ee7e"
server_key_version: "encryptionkey@0"
I20260812 11:10:29.267599 32082 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 11:10:29.270099 32100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:29.271736 32082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:10:29.271894 32082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "8eee5fdbfe18417999652c900f3bee36"
format_stamp: "Formatted at 2026-08-12 11:10:29 on dist-test-slave-drl0"
server_key: "8e809239e8e9954c80096557967a863e"
server_key_iv: "4d8ad0d0d362ffde6d2567be0a43ee7e"
server_key_version: "encryptionkey@0"
I20260812 11:10:29.272017 32082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:29.287416 32082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:29.288103 32082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:29.288302 32082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:29.298069 32176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:46383 every 8 connection(s)
I20260812 11:10:29.298079 32082 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:46383
I20260812 11:10:29.300495 32082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:10:29.301512 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 32082
I20260812 11:10:29.301620 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:10:29.301956 17587 external_mini_cluster.cc:1428] Setting key a4aab813c2c3bf66aa234f7dbc50ac14
I20260812 11:10:29.304910 32177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:29.312659 32177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8eee5fdbfe18417999652c900f3bee36: Bootstrap starting.
I20260812 11:10:29.314257 32180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:29.303438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57168 (local address 127.17.44.254:46383)
0812 11:10:29.304174 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:29.304197 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:29.304244 (+ 47us) server_negotiation.cc:407] Connection header received
0812 11:10:29.305235 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:29.305260 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:29.305801 (+ 541us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:29.306184 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:29.306673 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:29.308355 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:29.309276 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:29.310049 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:29.310250 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:29.311857 (+ 1607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:29.311864 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:29.311962 (+ 98us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:10:29.312500 (+ 538us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:29.312576 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:10:29.312602 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:29.312732 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:29.313159 (+ 427us) server_negotiation.cc:299] Negotiation successful
0812 11:10:29.313531 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":126,"threads_started":1}
I20260812 11:10:29.315157 32177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8eee5fdbfe18417999652c900f3bee36: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:29.316107 32177 log.cc:824] T 00000000000000000000000000000000 P 8eee5fdbfe18417999652c900f3bee36: Log is configured to *not* fsync() on all Append() calls
I20260812 11:10:29.318009 32177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8eee5fdbfe18417999652c900f3bee36: No bootstrap required, opened a new log
I20260812 11:10:29.321466 32177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8eee5fdbfe18417999652c900f3bee36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eee5fdbfe18417999652c900f3bee36" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 46383 } }
I20260812 11:10:29.321687 32177 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8eee5fdbfe18417999652c900f3bee36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:29.321723 32177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8eee5fdbfe18417999652c900f3bee36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eee5fdbfe18417999652c900f3bee36, State: Initialized, Role: FOLLOWER
I20260812 11:10:29.322232 32177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8eee5fdbfe18417999652c900f3bee36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eee5fdbfe18417999652c900f3bee36" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 46383 } }
I20260812 11:10:29.322405 32177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8eee5fdbfe18417999652c900f3bee36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:10:29.322515 32177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8eee5fdbfe18417999652c900f3bee36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:10:29.322635 32177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8eee5fdbfe18417999652c900f3bee36 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:29.323778 32177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8eee5fdbfe18417999652c900f3bee36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eee5fdbfe18417999652c900f3bee36" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 46383 } }
I20260812 11:10:29.324133 32177 leader_election.cc:302] T 00000000000000000000000000000000 P 8eee5fdbfe18417999652c900f3bee36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8eee5fdbfe18417999652c900f3bee36; no voters:
I20260812 11:10:29.324733 32177 leader_election.cc:288] T 00000000000000000000000000000000 P 8eee5fdbfe18417999652c900f3bee36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:10:29.324903 32183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8eee5fdbfe18417999652c900f3bee36 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:10:29.325129 32183 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8eee5fdbfe18417999652c900f3bee36 [term 1 LEADER]: Becoming Leader. State: Replica: 8eee5fdbfe18417999652c900f3bee36, State: Running, Role: LEADER
I20260812 11:10:29.325480 32183 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8eee5fdbfe18417999652c900f3bee36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eee5fdbfe18417999652c900f3bee36" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 46383 } }
I20260812 11:10:29.326089 32177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8eee5fdbfe18417999652c900f3bee36 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:10:29.327154 32188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eee5fdbfe18417999652c900f3bee36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8eee5fdbfe18417999652c900f3bee36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eee5fdbfe18417999652c900f3bee36" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 46383 } } }
I20260812 11:10:29.327291 32188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eee5fdbfe18417999652c900f3bee36 [sys.catalog]: This master's current role is: LEADER
I20260812 11:10:29.327574 32190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eee5fdbfe18417999652c900f3bee36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8eee5fdbfe18417999652c900f3bee36. Latest consensus state: current_term: 1 leader_uuid: "8eee5fdbfe18417999652c900f3bee36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eee5fdbfe18417999652c900f3bee36" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 46383 } } }
I20260812 11:10:29.327680 32203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:10:29.327733 32190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eee5fdbfe18417999652c900f3bee36 [sys.catalog]: This master's current role is: LEADER
I20260812 11:10:29.331023 32203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:10:29.338464 32203 catalog_manager.cc:1347] Generated new cluster ID: e1ab143978eb4f59bfc8f1cd2f6d9363
I20260812 11:10:29.338543 32203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:10:29.356352 32203 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:10:29.357316 32203 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:10:29.367233 32203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8eee5fdbfe18417999652c900f3bee36: Generated new TSK 0
I20260812 11:10:29.368029 32203 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:10:29.374332 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:46383
--builtin_ntp_servers=127.17.44.212:41013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 11:10:29.515877 32219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:29.516130 32219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:29.516166 32219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:29.520375 32219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:29.520439 32219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:29.520493 32219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:10:29.526420 32219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:41013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:46383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:29.528113 32219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:29.529966 32219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 11:10:29.541738 32219 server_base.cc:1034] running on GCE node
W20260812 11:10:29.541725 32231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:29.541720 32227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:29.542050 32228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:29.542727 32219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:29.543643 32219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:29.544955 32219 hybrid_clock.cc:648] HybridClock initialized: now 1786533029544843 us; error 66 us; skew 500 ppm
I20260812 11:10:29.547086 32219 webserver.cc:460] Webserver started at http://127.17.44.193:42663/ using document root <none> and password file <none>
I20260812 11:10:29.547732 32219 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:29.547832 32219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:29.548091 32219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:10:29.550213 32219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "b2ec8ff98f3e43e5bce00578589622e7"
format_stamp: "Formatted at 2026-08-12 11:10:29 on dist-test-slave-drl0"
server_key: "dbac90d3f4e898963424faf5be2bf10f"
server_key_iv: "86df832fe03636aa3fc3f976cbe85fb3"
server_key_version: "encryptionkey@0"
I20260812 11:10:29.550786 32219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "b2ec8ff98f3e43e5bce00578589622e7"
format_stamp: "Formatted at 2026-08-12 11:10:29 on dist-test-slave-drl0"
server_key: "dbac90d3f4e898963424faf5be2bf10f"
server_key_iv: "86df832fe03636aa3fc3f976cbe85fb3"
server_key_version: "encryptionkey@0"
I20260812 11:10:29.555176 32219 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260812 11:10:29.559108 32240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:29.560740 32219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 11:10:29.561012 32219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "b2ec8ff98f3e43e5bce00578589622e7"
format_stamp: "Formatted at 2026-08-12 11:10:29 on dist-test-slave-drl0"
server_key: "dbac90d3f4e898963424faf5be2bf10f"
server_key_iv: "86df832fe03636aa3fc3f976cbe85fb3"
server_key_version: "encryptionkey@0"
I20260812 11:10:29.561184 32219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:29.581698 32219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:29.582748 32219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:29.583050 32219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:29.583689 32219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:10:29.584721 32219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:10:29.584802 32219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:29.584877 32219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:10:29.584928 32219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:29.594687 32219 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:45531
I20260812 11:10:29.594722 32415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:45531 every 8 connection(s)
I20260812 11:10:29.595664 32219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:10:29.604135 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 32219
I20260812 11:10:29.604297 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:10:29.604612 17587 external_mini_cluster.cc:1428] Setting key f186baf9dec2b2bc1e0ed0df9401db25
I20260812 11:10:29.605932 32180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:29.597701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:50673 (local address 127.17.44.254:46383)
0812 11:10:29.597852 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:29.597856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:29.599094 (+ 1238us) server_negotiation.cc:407] Connection header received
0812 11:10:29.600550 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:29.600554 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:29.600616 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:29.600711 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:29.601662 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:29.602345 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:29.603745 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:29.603983 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:29.604106 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:29.605312 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:29.605315 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:29.605318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:10:29.605332 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:29.605358 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:10:29.605368 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:29.605450 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:29.605690 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 11:10:29.605782 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 11:10:29.606693 32418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:29.598152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:46383 (local address 127.17.44.193:50673)
0812 11:10:29.598865 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:29.598911 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:29.600335 (+ 1424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:29.600905 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:29.600911 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:29.601013 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:29.601515 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:29.601527 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:29.602609 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:29.602615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:29.603606 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:29.603617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:29.604150 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:29.604154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:29.604366 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:29.605051 (+ 685us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:10:29.605053 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:29.605090 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:10:29.605152 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:29.605451 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:29.605463 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:29.605472 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 11:10:29.605621 (+ 149us) client_negotiation.cc:240] Negotiation successful
0812 11:10:29.605874 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":144,"threads_started":1}
I20260812 11:10:29.607285 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:46383
--builtin_ntp_servers=127.17.44.212:41013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 11:10:29.607944 32416 heartbeater.cc:347] Connected to a master server at 127.17.44.254:46383
I20260812 11:10:29.608198 32416 heartbeater.cc:464] Registering TS with master...
I20260812 11:10:29.608825 32416 heartbeater.cc:511] Master 127.17.44.254:46383 requested a full tablet report, sending...
I20260812 11:10:29.611230 32123 ts_manager.cc:194] Registered new tserver with Master: b2ec8ff98f3e43e5bce00578589622e7 (127.17.44.193:45531)
I20260812 11:10:29.613229 32123 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.44.193:50673
W20260812 11:10:29.730049 32422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:29.730314 32422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:29.730352 32422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:29.734570 32422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:29.734627 32422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:29.734678 32422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:10:29.739424 32422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:41013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:46383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:29.740507 32422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:29.741863 32422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:29.753338 32432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:29.753407 32434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:29.753415 32430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:29.753580 32422 server_base.cc:1034] running on GCE node
I20260812 11:10:29.754036 32422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:29.754678 32422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:29.755875 32422 hybrid_clock.cc:648] HybridClock initialized: now 1786533029755864 us; error 70 us; skew 500 ppm
I20260812 11:10:29.757927 32422 webserver.cc:460] Webserver started at http://127.17.44.194:42501/ using document root <none> and password file <none>
I20260812 11:10:29.758563 32422 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:29.758634 32422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:29.758852 32422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:10:29.760967 32422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "2d6e1db320f04eef9e9664f41beea2f1"
format_stamp: "Formatted at 2026-08-12 11:10:29 on dist-test-slave-drl0"
server_key: "380c4fc2462ba51d8ea3e5aff8bab893"
server_key_iv: "d96392bc7d4621529497303e51478ebe"
server_key_version: "encryptionkey@0"
I20260812 11:10:29.761550 32422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "2d6e1db320f04eef9e9664f41beea2f1"
format_stamp: "Formatted at 2026-08-12 11:10:29 on dist-test-slave-drl0"
server_key: "380c4fc2462ba51d8ea3e5aff8bab893"
server_key_iv: "d96392bc7d4621529497303e51478ebe"
server_key_version: "encryptionkey@0"
I20260812 11:10:29.765584 32422 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 11:10:29.768640 32446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:29.770020 32422 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 11:10:29.770190 32422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "2d6e1db320f04eef9e9664f41beea2f1"
format_stamp: "Formatted at 2026-08-12 11:10:29 on dist-test-slave-drl0"
server_key: "380c4fc2462ba51d8ea3e5aff8bab893"
server_key_iv: "d96392bc7d4621529497303e51478ebe"
server_key_version: "encryptionkey@0"
I20260812 11:10:29.770346 32422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:29.784516 32422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:29.785215 32422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:29.785430 32422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:29.786041 32422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:10:29.787312 32422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:10:29.787391 32422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:29.787460 32422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:10:29.787509 32422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:29.799852 32422 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:33447
I20260812 11:10:29.799876 32614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:33447 every 8 connection(s)
I20260812 11:10:29.800755 32422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:10:29.807008 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 32422
I20260812 11:10:29.807132 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:10:29.807482 17587 external_mini_cluster.cc:1428] Setting key 122665e86c018f37a489cf85d29092b9
I20260812 11:10:29.810087 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:46383
--builtin_ntp_servers=127.17.44.212:41013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 11:10:29.813318 32180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:29.803097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:50671 (local address 127.17.44.254:46383)
0812 11:10:29.803291 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:29.803296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:29.804347 (+ 1051us) server_negotiation.cc:407] Connection header received
0812 11:10:29.806326 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:29.806330 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:29.806404 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:29.806514 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:29.807801 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:29.808487 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:29.810709 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:29.810944 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:29.811170 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:29.812671 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:29.812677 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:29.812683 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:10:29.812700 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:29.812727 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:10:29.812741 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:29.812829 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:29.813057 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 11:10:29.813136 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 11:10:29.814221 32618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:29.803260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:46383 (local address 127.17.44.194:50671)
0812 11:10:29.803972 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:29.804156 (+ 184us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:29.805951 (+ 1795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:29.806732 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:29.806742 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:29.806855 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:29.807621 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:29.807644 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:29.808692 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:29.808697 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:29.810546 (+ 1849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:29.810564 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:29.811234 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:29.811241 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:29.811494 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:29.812395 (+ 901us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:10:29.812396 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:29.812445 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:10:29.812509 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:29.812889 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:29.812898 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:29.812904 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 11:10:29.813028 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 11:10:29.813264 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":124,"threads_started":1}
I20260812 11:10:29.815604 32616 heartbeater.cc:347] Connected to a master server at 127.17.44.254:46383
I20260812 11:10:29.815877 32616 heartbeater.cc:464] Registering TS with master...
I20260812 11:10:29.816560 32616 heartbeater.cc:511] Master 127.17.44.254:46383 requested a full tablet report, sending...
I20260812 11:10:29.818188 32123 ts_manager.cc:194] Registered new tserver with Master: 2d6e1db320f04eef9e9664f41beea2f1 (127.17.44.194:33447)
I20260812 11:10:29.818823 32123 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.44.194:50671
W20260812 11:10:29.956106 32619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:29.956791 32619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:29.956893 32619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:29.965420 32619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:29.965502 32619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:29.965574 32619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:10:29.971035 32619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:41013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:46383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:29.972417 32619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:29.974038 32619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:29.985663 32627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:29.985697 32628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:29.985661 32633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:29.985944 32619 server_base.cc:1034] running on GCE node
I20260812 11:10:29.986508 32619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:29.987396 32619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:29.988602 32619 hybrid_clock.cc:648] HybridClock initialized: now 1786533029988564 us; error 51 us; skew 500 ppm
I20260812 11:10:29.991109 32619 webserver.cc:460] Webserver started at http://127.17.44.195:42387/ using document root <none> and password file <none>
I20260812 11:10:29.991741 32619 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:29.991811 32619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:29.992044 32619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:10:29.994290 32619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "b5e8039bea884db0a1dcb43d6f4471f9"
format_stamp: "Formatted at 2026-08-12 11:10:29 on dist-test-slave-drl0"
server_key: "6c24f7659b9062e8610b115b1a47ea09"
server_key_iv: "fd3ab3897330685b5f4104fa917ae86e"
server_key_version: "encryptionkey@0"
I20260812 11:10:29.995033 32619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "b5e8039bea884db0a1dcb43d6f4471f9"
format_stamp: "Formatted at 2026-08-12 11:10:29 on dist-test-slave-drl0"
server_key: "6c24f7659b9062e8610b115b1a47ea09"
server_key_iv: "fd3ab3897330685b5f4104fa917ae86e"
server_key_version: "encryptionkey@0"
I20260812 11:10:30.000469 32619 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260812 11:10:30.003552 32642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:30.004952 32619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:10:30.005090 32619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "b5e8039bea884db0a1dcb43d6f4471f9"
format_stamp: "Formatted at 2026-08-12 11:10:29 on dist-test-slave-drl0"
server_key: "6c24f7659b9062e8610b115b1a47ea09"
server_key_iv: "fd3ab3897330685b5f4104fa917ae86e"
server_key_version: "encryptionkey@0"
I20260812 11:10:30.005219 32619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:30.017938 32619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:30.018617 32619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:30.018834 32619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:30.019496 32619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:10:30.020633 32619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:10:30.020699 32619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:30.020771 32619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:10:30.020810 32619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:30.031322 32619 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:46305
I20260812 11:10:30.031375 348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:46305 every 8 connection(s)
I20260812 11:10:30.032341 32619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:10:30.041011 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 32619
I20260812 11:10:30.041134 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:10:30.041591 17587 external_mini_cluster.cc:1428] Setting key 460edd4fb1ba48c24b213b71306dc023
I20260812 11:10:30.046018 32180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:30.034379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:56003 (local address 127.17.44.254:46383)
0812 11:10:30.034570 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:30.034575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:30.036233 (+ 1658us) server_negotiation.cc:407] Connection header received
0812 11:10:30.037835 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:30.037838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:30.037889 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:30.037981 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:30.039717 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:30.040331 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:30.042818 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:30.043160 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:30.043302 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:30.045391 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:30.045394 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:30.045397 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:10:30.045412 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:30.045440 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:10:30.045450 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:30.045535 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:30.045837 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 11:10:30.045901 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 11:10:30.047072 352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:30.034793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:46383 (local address 127.17.44.195:56003)
0812 11:10:30.035915 (+ 1122us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:30.036071 (+ 156us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:30.037605 (+ 1534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:30.038326 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:30.038335 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:30.038575 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:30.039375 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:30.039397 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:30.040656 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:30.040664 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:30.042589 (+ 1925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:30.042607 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:30.043495 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:30.043500 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:30.043867 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:30.044842 (+ 975us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:10:30.044844 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:30.044946 (+ 102us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:10:30.045075 (+ 129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:30.045587 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:30.045601 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:30.045607 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 11:10:30.045836 (+ 229us) client_negotiation.cc:240] Negotiation successful
0812 11:10:30.046106 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":178,"threads_started":1}
I20260812 11:10:30.048556 349 heartbeater.cc:347] Connected to a master server at 127.17.44.254:46383
I20260812 11:10:30.048931 349 heartbeater.cc:464] Registering TS with master...
I20260812 11:10:30.049754 349 heartbeater.cc:511] Master 127.17.44.254:46383 requested a full tablet report, sending...
I20260812 11:10:30.051167 32123 ts_manager.cc:194] Registered new tserver with Master: b5e8039bea884db0a1dcb43d6f4471f9 (127.17.44.195:46305)
I20260812 11:10:30.051846 32123 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.44.195:56003
I20260812 11:10:30.056977 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 11:10:30.060299 32180 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:30.059105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57182 (local address 127.17.44.254:46383)
0812 11:10:30.059269 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:30.059276 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:30.059575 (+ 299us) server_negotiation.cc:407] Connection header received
0812 11:10:30.059753 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:30.059757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:30.059807 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:30.059909 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:30.060174 (+ 265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57182: BlockingRecv error: recv got EOF from 127.0.0.1:57182 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 11:10:30.060462 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 32219
I20260812 11:10:30.097076 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 32422
I20260812 11:10:30.113511 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 32619
I20260812 11:10:30.128719 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 32082
2026-08-12T11:10:30Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (1071 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T11:10:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:10:30Z Disabled control of system clock
I20260812 11:10:30.171665 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:44645
--webserver_interface=127.17.44.254
--webserver_port=0
--builtin_ntp_servers=127.17.44.212:34891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:44645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 11:10:30.303004 366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:30.303403 366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:30.303449 366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:30.308660 366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:10:30.308779 366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:30.308795 366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:30.308861 366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:10:30.308897 366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:10:30.315145 366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:34891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:44645
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:44645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:30.316776 366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:30.318883 366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:30.329312 375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:30.329313 378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:30.329512 376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:30.329921 366 server_base.cc:1034] running on GCE node
I20260812 11:10:30.330876 366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:30.332685 366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:30.333885 366 hybrid_clock.cc:648] HybridClock initialized: now 1786533030333860 us; error 112 us; skew 500 ppm
I20260812 11:10:30.336155 366 webserver.cc:460] Webserver started at http://127.17.44.254:34281/ using document root <none> and password file <none>
I20260812 11:10:30.336815 366 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:30.336921 366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:30.337169 366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:10:30.339406 366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "bb2c23d812b24c06b6707dc4189297b4"
format_stamp: "Formatted at 2026-08-12 11:10:30 on dist-test-slave-drl0"
server_key: "1b3b0944fc7d0238e44050bac2ee3805"
server_key_iv: "c164ba06e00ade8fe0186e3e6a70b4ce"
server_key_version: "encryptionkey@0"
I20260812 11:10:30.340046 366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "bb2c23d812b24c06b6707dc4189297b4"
format_stamp: "Formatted at 2026-08-12 11:10:30 on dist-test-slave-drl0"
server_key: "1b3b0944fc7d0238e44050bac2ee3805"
server_key_iv: "c164ba06e00ade8fe0186e3e6a70b4ce"
server_key_version: "encryptionkey@0"
I20260812 11:10:30.344024 366 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 11:10:30.347234 384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:30.349020 366 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260812 11:10:30.349196 366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "bb2c23d812b24c06b6707dc4189297b4"
format_stamp: "Formatted at 2026-08-12 11:10:30 on dist-test-slave-drl0"
server_key: "1b3b0944fc7d0238e44050bac2ee3805"
server_key_iv: "c164ba06e00ade8fe0186e3e6a70b4ce"
server_key_version: "encryptionkey@0"
I20260812 11:10:30.349337 366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:30.354259 366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:30.354880 366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:30.355099 366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:30.362854 366 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:44645
I20260812 11:10:30.362920 472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:44645 every 8 connection(s)
I20260812 11:10:30.363829 366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:10:30.366905 473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:30.370162 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 366
I20260812 11:10:30.370329 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:10:30.370630 17587 external_mini_cluster.cc:1428] Setting key 3111236ed6572812ce6a7a90e8c4122f
I20260812 11:10:30.373034 473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb2c23d812b24c06b6707dc4189297b4: Bootstrap starting.
I20260812 11:10:30.375342 473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb2c23d812b24c06b6707dc4189297b4: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:30.376099 473 log.cc:824] T 00000000000000000000000000000000 P bb2c23d812b24c06b6707dc4189297b4: Log is configured to *not* fsync() on all Append() calls
I20260812 11:10:30.376369 476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:30.372115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41638 (local address 127.17.44.254:44645)
0812 11:10:30.372807 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:30.372822 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:30.372856 (+ 34us) server_negotiation.cc:407] Connection header received
0812 11:10:30.373539 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:30.373556 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:30.373886 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:30.374215 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:30.375042 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:30.375047 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:30.375058 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:10:30.375069 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:30.375121 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:10:30.375137 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:30.375257 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:30.375431 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 11:10:30.375683 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":134,"threads_started":1}
I20260812 11:10:30.378266 473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb2c23d812b24c06b6707dc4189297b4: No bootstrap required, opened a new log
I20260812 11:10:30.382074 473 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb2c23d812b24c06b6707dc4189297b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb2c23d812b24c06b6707dc4189297b4" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44645 } }
I20260812 11:10:30.382287 473 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb2c23d812b24c06b6707dc4189297b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:30.382364 473 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb2c23d812b24c06b6707dc4189297b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb2c23d812b24c06b6707dc4189297b4, State: Initialized, Role: FOLLOWER
I20260812 11:10:30.382773 473 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb2c23d812b24c06b6707dc4189297b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb2c23d812b24c06b6707dc4189297b4" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44645 } }
I20260812 11:10:30.382895 473 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb2c23d812b24c06b6707dc4189297b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:10:30.382995 473 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb2c23d812b24c06b6707dc4189297b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:10:30.383096 473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb2c23d812b24c06b6707dc4189297b4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:30.383977 473 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb2c23d812b24c06b6707dc4189297b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb2c23d812b24c06b6707dc4189297b4" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44645 } }
I20260812 11:10:30.384325 473 leader_election.cc:302] T 00000000000000000000000000000000 P bb2c23d812b24c06b6707dc4189297b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb2c23d812b24c06b6707dc4189297b4; no voters:
I20260812 11:10:30.384629 473 leader_election.cc:288] T 00000000000000000000000000000000 P bb2c23d812b24c06b6707dc4189297b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:10:30.384868 479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb2c23d812b24c06b6707dc4189297b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:10:30.385267 479 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb2c23d812b24c06b6707dc4189297b4 [term 1 LEADER]: Becoming Leader. State: Replica: bb2c23d812b24c06b6707dc4189297b4, State: Running, Role: LEADER
I20260812 11:10:30.385573 479 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb2c23d812b24c06b6707dc4189297b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb2c23d812b24c06b6707dc4189297b4" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44645 } }
I20260812 11:10:30.385887 473 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb2c23d812b24c06b6707dc4189297b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:10:30.387189 481 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb2c23d812b24c06b6707dc4189297b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb2c23d812b24c06b6707dc4189297b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb2c23d812b24c06b6707dc4189297b4" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44645 } } }
I20260812 11:10:30.387233 483 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb2c23d812b24c06b6707dc4189297b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb2c23d812b24c06b6707dc4189297b4. Latest consensus state: current_term: 1 leader_uuid: "bb2c23d812b24c06b6707dc4189297b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb2c23d812b24c06b6707dc4189297b4" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44645 } } }
I20260812 11:10:30.387306 481 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb2c23d812b24c06b6707dc4189297b4 [sys.catalog]: This master's current role is: LEADER
I20260812 11:10:30.387333 483 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb2c23d812b24c06b6707dc4189297b4 [sys.catalog]: This master's current role is: LEADER
I20260812 11:10:30.387707 492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:10:30.390311 492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:10:30.395852 492 catalog_manager.cc:1347] Generated new cluster ID: 2ee6ef2b2ac744a09ae7b70b93ab4ef4
I20260812 11:10:30.395923 492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:10:30.414131 492 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:10:30.414512 492 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:10:30.424206 492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb2c23d812b24c06b6707dc4189297b4: Generated new TSK 0
I20260812 11:10:30.425396 492 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:10:30.433943 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:44645
--builtin_ntp_servers=127.17.44.212:34891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 11:10:30.562114 508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:30.562448 508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:30.562491 508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:30.567028 508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:30.567088 508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:30.567143 508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:10:30.572222 508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:34891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:44645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:30.573482 508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:30.574903 508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:30.585290 519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:30.585319 515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:30.585393 517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:30.585638 508 server_base.cc:1034] running on GCE node
I20260812 11:10:30.586072 508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:30.586906 508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:30.588230 508 hybrid_clock.cc:648] HybridClock initialized: now 1786533030588185 us; error 40 us; skew 500 ppm
I20260812 11:10:30.590744 508 webserver.cc:460] Webserver started at http://127.17.44.193:33239/ using document root <none> and password file <none>
I20260812 11:10:30.591403 508 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:30.591476 508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:30.591697 508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:10:30.593883 508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "d5f29d67f41f4485ad6e2890392a51e7"
format_stamp: "Formatted at 2026-08-12 11:10:30 on dist-test-slave-drl0"
server_key: "44077ae307cf8c872d23e18a97ec63ad"
server_key_iv: "acd679c69f94962058da554b41a96da4"
server_key_version: "encryptionkey@0"
I20260812 11:10:30.594468 508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "d5f29d67f41f4485ad6e2890392a51e7"
format_stamp: "Formatted at 2026-08-12 11:10:30 on dist-test-slave-drl0"
server_key: "44077ae307cf8c872d23e18a97ec63ad"
server_key_iv: "acd679c69f94962058da554b41a96da4"
server_key_version: "encryptionkey@0"
I20260812 11:10:30.598100 508 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260812 11:10:30.600741 528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:30.602222 508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 11:10:30.602455 508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "d5f29d67f41f4485ad6e2890392a51e7"
format_stamp: "Formatted at 2026-08-12 11:10:30 on dist-test-slave-drl0"
server_key: "44077ae307cf8c872d23e18a97ec63ad"
server_key_iv: "acd679c69f94962058da554b41a96da4"
server_key_version: "encryptionkey@0"
I20260812 11:10:30.602579 508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:30.607232 508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:30.607842 508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:30.608034 508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:30.608595 508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:10:30.609548 508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:10:30.609624 508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:30.609684 508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:10:30.609733 508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:30.620404 508 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:45839
I20260812 11:10:30.620527 698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:45839 every 8 connection(s)
I20260812 11:10:30.621361 508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:10:30.623896 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 508
I20260812 11:10:30.624058 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:10:30.624452 17587 external_mini_cluster.cc:1428] Setting key 6e2d50c92de5a6ad0709cba0bdc64987
I20260812 11:10:30.627090 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:44645
--builtin_ntp_servers=127.17.44.212:34891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 11:10:30.630569 476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:30.624199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:52575 (local address 127.17.44.254:44645)
0812 11:10:30.624360 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:30.624365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:30.626034 (+ 1669us) server_negotiation.cc:407] Connection header received
0812 11:10:30.628434 (+ 2400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:30.628437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:30.628476 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:30.628552 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:30.629985 (+ 1433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:30.629991 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:30.629997 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:10:30.630014 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:30.630041 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:10:30.630054 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:30.630163 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:30.630348 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 11:10:30.630400 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 11:10:30.631490 701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:30.624835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:44645 (local address 127.17.44.193:52575)
0812 11:10:30.625825 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:30.625887 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:30.628085 (+ 2198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:30.628789 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:30.628802 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:30.628935 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:30.629705 (+ 770us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:10:30.629707 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:30.629748 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:10:30.629820 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:30.630182 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:30.630191 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:30.630197 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 11:10:30.630361 (+ 164us) client_negotiation.cc:240] Negotiation successful
0812 11:10:30.630579 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":234,"threads_started":1}
I20260812 11:10:30.633267 699 heartbeater.cc:347] Connected to a master server at 127.17.44.254:44645
I20260812 11:10:30.633540 699 heartbeater.cc:464] Registering TS with master...
I20260812 11:10:30.633991 699 heartbeater.cc:511] Master 127.17.44.254:44645 requested a full tablet report, sending...
I20260812 11:10:30.635906 415 ts_manager.cc:194] Registered new tserver with Master: d5f29d67f41f4485ad6e2890392a51e7 (127.17.44.193:45839)
W20260812 11:10:30.751477 704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:30.751741 704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:30.751782 704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:30.756791 704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:30.756857 704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:30.756920 704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:10:30.764304 704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:34891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:44645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:30.765758 704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:30.767633 704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 11:10:30.779227 704 server_base.cc:1034] running on GCE node
W20260812 11:10:30.779170 715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:30.779160 711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:30.779170 713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:30.780141 704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:30.781152 704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:30.782369 704 hybrid_clock.cc:648] HybridClock initialized: now 1786533030782348 us; error 40 us; skew 500 ppm
I20260812 11:10:30.784551 704 webserver.cc:460] Webserver started at http://127.17.44.194:37913/ using document root <none> and password file <none>
I20260812 11:10:30.785157 704 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:30.785262 704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:30.785497 704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:10:30.787899 704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "a03b2e6222d1449db6b1d40f37e528d6"
format_stamp: "Formatted at 2026-08-12 11:10:30 on dist-test-slave-drl0"
server_key: "93b25abd4c2de9de1b7ac5d36daeaaf2"
server_key_iv: "a320fdadbe6924af4f34f72e625a330f"
server_key_version: "encryptionkey@0"
I20260812 11:10:30.788460 704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "a03b2e6222d1449db6b1d40f37e528d6"
format_stamp: "Formatted at 2026-08-12 11:10:30 on dist-test-slave-drl0"
server_key: "93b25abd4c2de9de1b7ac5d36daeaaf2"
server_key_iv: "a320fdadbe6924af4f34f72e625a330f"
server_key_version: "encryptionkey@0"
I20260812 11:10:30.792446 704 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 11:10:30.795544 722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:30.797456 704 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:10:30.797618 704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "a03b2e6222d1449db6b1d40f37e528d6"
format_stamp: "Formatted at 2026-08-12 11:10:30 on dist-test-slave-drl0"
server_key: "93b25abd4c2de9de1b7ac5d36daeaaf2"
server_key_iv: "a320fdadbe6924af4f34f72e625a330f"
server_key_version: "encryptionkey@0"
I20260812 11:10:30.797740 704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:30.803531 704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:30.804154 704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:30.804348 704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:30.804889 704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:10:30.805884 704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:10:30.805955 704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:30.806012 704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:10:30.806059 704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:30.816057 704 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:44355
I20260812 11:10:30.816115 894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:44355 every 8 connection(s)
I20260812 11:10:30.817077 704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:10:30.823945 476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:30.819477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:57305 (local address 127.17.44.254:44645)
0812 11:10:30.819633 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:30.819638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:30.820869 (+ 1231us) server_negotiation.cc:407] Connection header received
0812 11:10:30.822165 (+ 1296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:30.822168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:30.822207 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:30.822297 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:30.823561 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:30.823564 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:30.823567 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:10:30.823578 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:30.823596 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:10:30.823603 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:30.823694 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:30.823793 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 11:10:30.823831 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 11:10:30.824852 901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:30.819965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:44645 (local address 127.17.44.194:57305)
0812 11:10:30.820690 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:30.820729 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:30.821961 (+ 1232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:30.822458 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:30.822464 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:30.822546 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:30.823272 (+ 726us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:10:30.823274 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:30.823313 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:10:30.823389 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:30.823679 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:30.823683 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:30.823687 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 11:10:30.823802 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 11:10:30.823995 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":122,"threads_started":1}
I20260812 11:10:30.826045 895 heartbeater.cc:347] Connected to a master server at 127.17.44.254:44645
I20260812 11:10:30.826246 895 heartbeater.cc:464] Registering TS with master...
I20260812 11:10:30.826594 895 heartbeater.cc:511] Master 127.17.44.254:44645 requested a full tablet report, sending...
I20260812 11:10:30.827260 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 704
I20260812 11:10:30.827430 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:10:30.827754 17587 external_mini_cluster.cc:1428] Setting key b99870976607c3f43150eff9478480d8
I20260812 11:10:30.827795 415 ts_manager.cc:194] Registered new tserver with Master: a03b2e6222d1449db6b1d40f37e528d6 (127.17.44.194:44355)
I20260812 11:10:30.829716 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:44645
--builtin_ntp_servers=127.17.44.212:34891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 11:10:30.954173 904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:30.954463 904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:30.954504 904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:30.958595 904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:30.958653 904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:30.958710 904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:10:30.963893 904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:34891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:44645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:30.965041 904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:30.966784 904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:30.978025 917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:30.978025 911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:30.978216 914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:30.978281 904 server_base.cc:1034] running on GCE node
I20260812 11:10:30.978751 904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:30.979456 904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:30.980635 904 hybrid_clock.cc:648] HybridClock initialized: now 1786533030980603 us; error 59 us; skew 500 ppm
I20260812 11:10:30.983223 904 webserver.cc:460] Webserver started at http://127.17.44.195:41907/ using document root <none> and password file <none>
I20260812 11:10:30.983817 904 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:30.983883 904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:30.984099 904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:10:30.986392 904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "5bea2003899e451fb9da7832be815dfd"
format_stamp: "Formatted at 2026-08-12 11:10:30 on dist-test-slave-drl0"
server_key: "d2a11a43df96ad2722530b7a57f2fb19"
server_key_iv: "13d8b038091f252419c1d40f0e46036b"
server_key_version: "encryptionkey@0"
I20260812 11:10:30.987112 904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "5bea2003899e451fb9da7832be815dfd"
format_stamp: "Formatted at 2026-08-12 11:10:30 on dist-test-slave-drl0"
server_key: "d2a11a43df96ad2722530b7a57f2fb19"
server_key_iv: "13d8b038091f252419c1d40f0e46036b"
server_key_version: "encryptionkey@0"
I20260812 11:10:30.991642 904 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 11:10:30.994376 924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:30.996071 904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:10:30.996174 904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "5bea2003899e451fb9da7832be815dfd"
format_stamp: "Formatted at 2026-08-12 11:10:30 on dist-test-slave-drl0"
server_key: "d2a11a43df96ad2722530b7a57f2fb19"
server_key_iv: "13d8b038091f252419c1d40f0e46036b"
server_key_version: "encryptionkey@0"
I20260812 11:10:30.996258 904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:31.000936 904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:31.001565 904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:31.001753 904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:31.002322 904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:10:31.003387 904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:10:31.003439 904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:31.003476 904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:10:31.003652 904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:31.014083 904 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:37531
I20260812 11:10:31.014114 1101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:37531 every 8 connection(s)
I20260812 11:10:31.015008 904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:10:31.019778 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 904
I20260812 11:10:31.019893 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:10:31.020313 17587 external_mini_cluster.cc:1428] Setting key f88b3069f5bc870d087921507dd8d133
I20260812 11:10:31.022066 476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:31.017083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:58573 (local address 127.17.44.254:44645)
0812 11:10:31.017310 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:31.017314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:31.018235 (+ 921us) server_negotiation.cc:407] Connection header received
0812 11:10:31.020195 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:31.020198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:31.020236 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:31.020307 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:31.021593 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:31.021597 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:31.021599 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:10:31.021612 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:31.021634 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:10:31.021644 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:31.021718 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:31.021896 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 11:10:31.021940 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260812 11:10:31.023001 1106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:31.017384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:44645 (local address 127.17.44.195:58573)
0812 11:10:31.018071 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:31.018110 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:31.019747 (+ 1637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:31.020525 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:31.020533 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:31.020641 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:31.021342 (+ 701us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:10:31.021344 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:31.021384 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:10:31.021450 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:31.021738 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:31.021749 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:31.021757 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 11:10:31.021909 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 11:10:31.022103 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":117,"threads_started":1}
I20260812 11:10:31.024247 1104 heartbeater.cc:347] Connected to a master server at 127.17.44.254:44645
I20260812 11:10:31.024461 1104 heartbeater.cc:464] Registering TS with master...
I20260812 11:10:31.024827 1104 heartbeater.cc:511] Master 127.17.44.254:44645 requested a full tablet report, sending...
I20260812 11:10:31.025811 415 ts_manager.cc:194] Registered new tserver with Master: 5bea2003899e451fb9da7832be815dfd (127.17.44.195:37531)
I20260812 11:10:31.034261 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 11:10:31.038425 476 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:10:31.037250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41654 (local address 127.17.44.254:44645)
0812 11:10:31.037470 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:31.037475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:31.037635 (+ 160us) server_negotiation.cc:407] Connection header received
0812 11:10:31.037916 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:31.037921 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:31.037965 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:31.038057 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:31.038313 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41654: BlockingRecv error: recv got EOF from 127.0.0.1:41654 (error 108)
Metrics: {"server-negotiator.queue_time_us":116}
I20260812 11:10:31.039609 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 508
I20260812 11:10:31.068195 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 704
I20260812 11:10:31.090432 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 904
I20260812 11:10:31.106273 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 366
2026-08-12T11:10:31Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (971 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:10:31 dist-test-slave-drl0 krb5kdc[1114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:10:31 dist-test-slave-drl0 krb5kdc[1114](info): set up 2 sockets
Aug 12 11:10:31 dist-test-slave-drl0 krb5kdc[1114](info): commencing operation
krb5kdc: starting...
W20260812 11:10:32.411237 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:10:32 dist-test-slave-drl0 krb5kdc[1114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:10:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:10:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:10:32.552497 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:43515
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:43515
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:10:32.684645 1136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:32.685051 1136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:32.685127 1136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:32.689131 1136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:10:32.689234 1136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:32.689278 1136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:32.689337 1136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:10:32.689378 1136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:10:32.694603 1136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:43515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:43515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:32.695758 1136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:32.696827 1136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:32.707738 1144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:32.707839 1136 server_base.cc:1034] running on GCE node
W20260812 11:10:32.707710 1143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:32.707729 1151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:32.708528 1136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:32.709661 1136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:32.710903 1136 hybrid_clock.cc:648] HybridClock initialized: now 1786533032710837 us; error 69 us; skew 500 ppm
Aug 12 11:10:32 dist-test-slave-drl0 krb5kdc[1114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533032, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:10:32.714046 1136 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:10:32.715225 1136 webserver.cc:460] Webserver started at http://127.17.44.254:45455/ using document root <none> and password file <none>
I20260812 11:10:32.715814 1136 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:32.715878 1136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:32.716115 1136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:10:32.718362 1136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "33ca74abc79243c5bd53be832bebae7c"
format_stamp: "Formatted at 2026-08-12 11:10:32 on dist-test-slave-drl0"
server_key: "4f865729ded279ea94e9ad5a61c86d8c"
server_key_iv: "bcd74a6eedbbce7346725e4aa3cab0a9"
server_key_version: "encryptionkey@0"
I20260812 11:10:32.719040 1136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "33ca74abc79243c5bd53be832bebae7c"
format_stamp: "Formatted at 2026-08-12 11:10:32 on dist-test-slave-drl0"
server_key: "4f865729ded279ea94e9ad5a61c86d8c"
server_key_iv: "bcd74a6eedbbce7346725e4aa3cab0a9"
server_key_version: "encryptionkey@0"
I20260812 11:10:32.723057 1136 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 11:10:32.725698 1166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:32.727187 1136 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:10:32.727293 1136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "33ca74abc79243c5bd53be832bebae7c"
format_stamp: "Formatted at 2026-08-12 11:10:32 on dist-test-slave-drl0"
server_key: "4f865729ded279ea94e9ad5a61c86d8c"
server_key_iv: "bcd74a6eedbbce7346725e4aa3cab0a9"
server_key_version: "encryptionkey@0"
I20260812 11:10:32.727424 1136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:32.738157 1136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:32.745530 1136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:32.745697 1136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:32.754593 1136 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:43515
I20260812 11:10:32.754670 1239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:43515 every 8 connection(s)
I20260812 11:10:32.757037 1136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:10:32.760131 1243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:32.763070 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 1136
I20260812 11:10:32.763242 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:10:32.763532 17587 external_mini_cluster.cc:1428] Setting key 65ac7d03f4f853c0bec387704be247a6
I20260812 11:10:32.768038 1243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33ca74abc79243c5bd53be832bebae7c: Bootstrap starting.
I20260812 11:10:32.770668 1243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33ca74abc79243c5bd53be832bebae7c: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:32.771932 1243 log.cc:824] T 00000000000000000000000000000000 P 33ca74abc79243c5bd53be832bebae7c: Log is configured to *not* fsync() on all Append() calls
Aug 12 11:10:32 dist-test-slave-drl0 krb5kdc[1114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:10:32.774537 1243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33ca74abc79243c5bd53be832bebae7c: No bootstrap required, opened a new log
I20260812 11:10:32.779246 1243 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33ca74abc79243c5bd53be832bebae7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ca74abc79243c5bd53be832bebae7c" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43515 } }
I20260812 11:10:32.779636 1243 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33ca74abc79243c5bd53be832bebae7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:32.779711 1243 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33ca74abc79243c5bd53be832bebae7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33ca74abc79243c5bd53be832bebae7c, State: Initialized, Role: FOLLOWER
I20260812 11:10:32.780411 1243 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33ca74abc79243c5bd53be832bebae7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ca74abc79243c5bd53be832bebae7c" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43515 } }
I20260812 11:10:32.780663 1243 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33ca74abc79243c5bd53be832bebae7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:10:32.780725 1248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:32.765330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33136 (local address 127.17.44.254:43515)
0812 11:10:32.765823 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:32.765842 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:32.765881 (+ 39us) server_negotiation.cc:407] Connection header received
0812 11:10:32.766741 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:32.766760 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:32.767152 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:32.767447 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:32.768435 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:32.769528 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:32.770731 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:32.771343 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:32.771467 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:32.775340 (+ 3873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:32.775345 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:32.775359 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:32.775390 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:32.777289 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:32.777948 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:32.777955 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:32.777964 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:32.778094 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:32.778418 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:32.778421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:32.778423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:32.778869 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:32.779185 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:32.779473 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 11:10:32.779848 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":166,"threads_started":1}
I20260812 11:10:32.780777 1243 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33ca74abc79243c5bd53be832bebae7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:10:32.780934 1243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33ca74abc79243c5bd53be832bebae7c [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:32.782054 1243 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33ca74abc79243c5bd53be832bebae7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ca74abc79243c5bd53be832bebae7c" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43515 } }
I20260812 11:10:32.782459 1243 leader_election.cc:302] T 00000000000000000000000000000000 P 33ca74abc79243c5bd53be832bebae7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33ca74abc79243c5bd53be832bebae7c; no voters:
I20260812 11:10:32.782864 1243 leader_election.cc:288] T 00000000000000000000000000000000 P 33ca74abc79243c5bd53be832bebae7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:10:32.783085 1250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33ca74abc79243c5bd53be832bebae7c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:10:32.783396 1250 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33ca74abc79243c5bd53be832bebae7c [term 1 LEADER]: Becoming Leader. State: Replica: 33ca74abc79243c5bd53be832bebae7c, State: Running, Role: LEADER
I20260812 11:10:32.783725 1250 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33ca74abc79243c5bd53be832bebae7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ca74abc79243c5bd53be832bebae7c" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43515 } }
I20260812 11:10:32.784415 1243 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33ca74abc79243c5bd53be832bebae7c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:10:32.785805 1252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33ca74abc79243c5bd53be832bebae7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33ca74abc79243c5bd53be832bebae7c. Latest consensus state: current_term: 1 leader_uuid: "33ca74abc79243c5bd53be832bebae7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ca74abc79243c5bd53be832bebae7c" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43515 } } }
I20260812 11:10:32.785806 1251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33ca74abc79243c5bd53be832bebae7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33ca74abc79243c5bd53be832bebae7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ca74abc79243c5bd53be832bebae7c" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43515 } } }
I20260812 11:10:32.785977 1252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33ca74abc79243c5bd53be832bebae7c [sys.catalog]: This master's current role is: LEADER
I20260812 11:10:32.785977 1251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33ca74abc79243c5bd53be832bebae7c [sys.catalog]: This master's current role is: LEADER
I20260812 11:10:32.787243 1262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:10:32.787654 1275 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 11:10:32.791563 1262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:10:32.798044 1262 catalog_manager.cc:1347] Generated new cluster ID: 3a7b974b28ce4a11b6cc9b27b55700ff
I20260812 11:10:32.798126 1262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:10:32.819098 1262 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:10:32.820262 1262 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:10:32.829656 1262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33ca74abc79243c5bd53be832bebae7c: Generated new TSK 0
I20260812 11:10:32.830536 1262 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
I20260812 11:10:32.865810 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:43515
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:10:33.006772 1287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:33.007113 1287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:33.007164 1287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:33.011410 1287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:33.011474 1287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:33.011531 1287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:10:33.021602 1287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:43515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:33.024065 1287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:33.025396 1287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:33.036242 1296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:33.036290 1295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:33.036413 1287 server_base.cc:1034] running on GCE node
W20260812 11:10:33.036576 1301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:33.036990 1287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:33.037737 1287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:33.039014 1287 hybrid_clock.cc:648] HybridClock initialized: now 1786533033038992 us; error 31 us; skew 500 ppm
Aug 12 11:10:33 dist-test-slave-drl0 krb5kdc[1114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533033, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:10:33.042593 1287 init.cc:377] Logged in from keytab as kudu/127.17.44.193@KRBTEST.COM (short username kudu)
I20260812 11:10:33.043529 1287 webserver.cc:460] Webserver started at http://127.17.44.193:43359/ using document root <none> and password file <none>
I20260812 11:10:33.044112 1287 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:33.044207 1287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:33.044437 1287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:10:33.046527 1287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "9f6e08d9e6c44711bfc88a5facf6fa5b"
format_stamp: "Formatted at 2026-08-12 11:10:33 on dist-test-slave-drl0"
server_key: "c5ec517874ff8705743cae3f65ce2cc5"
server_key_iv: "0a0c2994bbeec47bd0b115096a9422be"
server_key_version: "encryptionkey@0"
I20260812 11:10:33.047161 1287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "9f6e08d9e6c44711bfc88a5facf6fa5b"
format_stamp: "Formatted at 2026-08-12 11:10:33 on dist-test-slave-drl0"
server_key: "c5ec517874ff8705743cae3f65ce2cc5"
server_key_iv: "0a0c2994bbeec47bd0b115096a9422be"
server_key_version: "encryptionkey@0"
I20260812 11:10:33.050793 1287 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:10:33.053891 1311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:33.055820 1287 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 11:10:33.055968 1287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "9f6e08d9e6c44711bfc88a5facf6fa5b"
format_stamp: "Formatted at 2026-08-12 11:10:33 on dist-test-slave-drl0"
server_key: "c5ec517874ff8705743cae3f65ce2cc5"
server_key_iv: "0a0c2994bbeec47bd0b115096a9422be"
server_key_version: "encryptionkey@0"
I20260812 11:10:33.056164 1287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:33.070562 1287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:33.073805 1287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:33.073971 1287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:33.074513 1287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:10:33.075428 1287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:10:33.075488 1287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:33.075529 1287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:10:33.075544 1287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:33.086217 1287 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:39117
I20260812 11:10:33.086282 1481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:39117 every 8 connection(s)
I20260812 11:10:33.087713 1287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:10:33.095319 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 1287
I20260812 11:10:33.095467 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:10:33.095753 17587 external_mini_cluster.cc:1428] Setting key efc67b525ed5ad2f5e1684154fe406ef
Aug 12 11:10:33 dist-test-slave-drl0 krb5kdc[1114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533033, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:10:33.104684 1248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:33.089658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:40819 (local address 127.17.44.254:43515)
0812 11:10:33.089841 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:33.089846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:33.090656 (+ 810us) server_negotiation.cc:407] Connection header received
0812 11:10:33.092122 (+ 1466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:33.092126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:33.092184 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:33.092272 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:33.093603 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.094170 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.095524 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.095762 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.095908 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:33.099070 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:33.099073 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:33.099075 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:33.099099 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:33.101344 (+ 2245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:33.102363 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:33.102369 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:33.102374 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:33.102474 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:33.103005 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:33.103011 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:33.103016 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:33.103367 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:33.103504 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:33.104365 (+ 861us) server_negotiation.cc:299] Negotiation successful
0812 11:10:33.104501 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 11:10:33.105605 1484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:33.090009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:43515 (local address 127.17.44.193:40819)
0812 11:10:33.090491 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:33.090532 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:33.091909 (+ 1377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:33.092435 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:33.092441 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:33.092806 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:33.093435 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:33.093449 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.094314 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.094318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:33.095353 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:33.095365 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.096065 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.096068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:33.096287 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:33.097055 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:33.097074 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:33.098839 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:33.101853 (+ 3014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:33.101858 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:33.101869 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:33.102233 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:33.102596 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:33.102599 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:33.102600 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:33.102843 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:33.103625 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:33.103631 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:33.103896 (+ 265us) client_negotiation.cc:769] Sending connection context
0812 11:10:33.104312 (+ 416us) client_negotiation.cc:240] Negotiation successful
0812 11:10:33.104653 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":131,"threads_started":1}
I20260812 11:10:33.107084 1482 heartbeater.cc:347] Connected to a master server at 127.17.44.254:43515
I20260812 11:10:33.107348 1482 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.17.44.194@KRBTEST.COM; defaulting to no policy
I20260812 11:10:33.108397 1482 heartbeater.cc:511] Master 127.17.44.254:43515 requested a full tablet report, sending...
I20260812 11:10:33.110067 1190 ts_manager.cc:194] Registered new tserver with Master: 9f6e08d9e6c44711bfc88a5facf6fa5b (127.17.44.193:39117)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.194@KRBTEST.COM" created.
I20260812 11:10:33.111430 1190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.193@KRBTEST.COM'} at 127.17.44.193:40819
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
I20260812 11:10:33.130509 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:43515
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:10:33.258841 1490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:33.259135 1490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:33.259176 1490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:33.263520 1490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:33.263576 1490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:33.263633 1490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:10:33.268760 1490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:43515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:33.269826 1490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:33.270874 1490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:33.280541 1504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:33.280668 1500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:33.280537 1501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:33.281041 1490 server_base.cc:1034] running on GCE node
I20260812 11:10:33.281503 1490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:33.282250 1490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:33.283493 1490 hybrid_clock.cc:648] HybridClock initialized: now 1786533033283466 us; error 50 us; skew 500 ppm
Aug 12 11:10:33 dist-test-slave-drl0 krb5kdc[1114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533033, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:10:33.287031 1490 init.cc:377] Logged in from keytab as kudu/127.17.44.194@KRBTEST.COM (short username kudu)
I20260812 11:10:33.287955 1490 webserver.cc:460] Webserver started at http://127.17.44.194:35987/ using document root <none> and password file <none>
I20260812 11:10:33.288554 1490 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:33.288650 1490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:33.288898 1490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:10:33.290910 1490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "4e3307fc56314e39b1b88eecd76455d3"
format_stamp: "Formatted at 2026-08-12 11:10:33 on dist-test-slave-drl0"
server_key: "c993b73ea9f011b0575da1e1c293e40d"
server_key_iv: "cec5c1d94b4809e8bdcde34b31896c3e"
server_key_version: "encryptionkey@0"
I20260812 11:10:33.291554 1490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "4e3307fc56314e39b1b88eecd76455d3"
format_stamp: "Formatted at 2026-08-12 11:10:33 on dist-test-slave-drl0"
server_key: "c993b73ea9f011b0575da1e1c293e40d"
server_key_iv: "cec5c1d94b4809e8bdcde34b31896c3e"
server_key_version: "encryptionkey@0"
I20260812 11:10:33.297366 1490 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 11:10:33.300675 1513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:33.302175 1490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 11:10:33.302345 1490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "4e3307fc56314e39b1b88eecd76455d3"
format_stamp: "Formatted at 2026-08-12 11:10:33 on dist-test-slave-drl0"
server_key: "c993b73ea9f011b0575da1e1c293e40d"
server_key_iv: "cec5c1d94b4809e8bdcde34b31896c3e"
server_key_version: "encryptionkey@0"
I20260812 11:10:33.302471 1490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:33.325448 1490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:33.329169 1490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:33.329407 1490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:33.330114 1490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:10:33.331125 1490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:10:33.331422 1490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:33.331566 1490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:10:33.331729 1490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:33.342919 1490 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:46725
I20260812 11:10:33.343091 1699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:46725 every 8 connection(s)
I20260812 11:10:33.343869 1490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:10:33.349395 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 1490
I20260812 11:10:33.349519 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:10:33.349826 17587 external_mini_cluster.cc:1428] Setting key e3b99d1483da3b9a7d778bcbe8b9ce27
Aug 12 11:10:33 dist-test-slave-drl0 krb5kdc[1114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533033, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:10:33.360688 1248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:33.345769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:44083 (local address 127.17.44.254:43515)
0812 11:10:33.345963 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:33.345967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:33.346774 (+ 807us) server_negotiation.cc:407] Connection header received
0812 11:10:33.348429 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:33.348433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:33.348481 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:33.348561 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:33.350061 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.350696 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.352060 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.352278 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.352409 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:33.355629 (+ 3220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:33.355636 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:33.355642 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:33.355675 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:33.357658 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:33.358439 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:33.358443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:33.358446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:33.358543 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:33.359013 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:33.359017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:33.359019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:33.359281 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:33.359401 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:33.360323 (+ 922us) server_negotiation.cc:299] Negotiation successful
0812 11:10:33.360473 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
WARNING: no policy specified for kudu/127.17.44.195@KRBTEST.COM; defaulting to no policy
I20260812 11:10:33.362185 1703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:33.346138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:43515 (local address 127.17.44.194:44083)
0812 11:10:33.346612 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:33.346653 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:33.348141 (+ 1488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:33.348749 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:33.348755 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:33.349138 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:33.349832 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:33.349856 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.350840 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.350843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:33.351910 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:33.351920 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.352432 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.352436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:33.352629 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:33.353274 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:33.353306 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:33.355449 (+ 2143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:33.357868 (+ 2419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:33.357877 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:33.357898 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:33.358288 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:33.358668 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:33.358685 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:33.358687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:33.358853 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:33.359505 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:33.359513 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:33.359931 (+ 418us) client_negotiation.cc:769] Sending connection context
0812 11:10:33.360205 (+ 274us) client_negotiation.cc:240] Negotiation successful
0812 11:10:33.360816 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":126,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.195@KRBTEST.COM" created.
I20260812 11:10:33.364050 1700 heartbeater.cc:347] Connected to a master server at 127.17.44.254:43515
I20260812 11:10:33.364456 1700 heartbeater.cc:464] Registering TS with master...
I20260812 11:10:33.365110 1700 heartbeater.cc:511] Master 127.17.44.254:43515 requested a full tablet report, sending...
I20260812 11:10:33.366690 1190 ts_manager.cc:194] Registered new tserver with Master: 4e3307fc56314e39b1b88eecd76455d3 (127.17.44.194:46725)
I20260812 11:10:33.367560 1190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.194@KRBTEST.COM'} at 127.17.44.194:44083
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
I20260812 11:10:33.383291 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:43515
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:10:33.508735 1707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:10:33.509030 1707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:10:33.509104 1707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:10:33.513257 1707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:10:33.513341 1707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:10:33.513440 1707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:10:33.519536 1707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:43515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:10:33.521054 1707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:10:33.522696 1707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:10:33.534979 1719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:33.534914 1716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:10:33.534914 1717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:10:33.535264 1707 server_base.cc:1034] running on GCE node
I20260812 11:10:33.535715 1707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:10:33.536360 1707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:10:33.537599 1707 hybrid_clock.cc:648] HybridClock initialized: now 1786533033537540 us; error 70 us; skew 500 ppm
Aug 12 11:10:33 dist-test-slave-drl0 krb5kdc[1114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533033, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:10:33.541020 1707 init.cc:377] Logged in from keytab as kudu/127.17.44.195@KRBTEST.COM (short username kudu)
I20260812 11:10:33.541945 1707 webserver.cc:460] Webserver started at http://127.17.44.195:33957/ using document root <none> and password file <none>
I20260812 11:10:33.542539 1707 fs_manager.cc:359] Metadata directory not provided
I20260812 11:10:33.542639 1707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:10:33.542886 1707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:10:33.545027 1707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "9845eda5459e4861bbf9936ba040b245"
format_stamp: "Formatted at 2026-08-12 11:10:33 on dist-test-slave-drl0"
server_key: "0746c7f8c29388feb000ae686bb210d7"
server_key_iv: "cdaef28fd96be5c03724412ec07e4edb"
server_key_version: "encryptionkey@0"
I20260812 11:10:33.545621 1707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "9845eda5459e4861bbf9936ba040b245"
format_stamp: "Formatted at 2026-08-12 11:10:33 on dist-test-slave-drl0"
server_key: "0746c7f8c29388feb000ae686bb210d7"
server_key_iv: "cdaef28fd96be5c03724412ec07e4edb"
server_key_version: "encryptionkey@0"
I20260812 11:10:33.549896 1707 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 11:10:33.552515 1729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:33.553885 1707 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:10:33.554040 1707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "9845eda5459e4861bbf9936ba040b245"
format_stamp: "Formatted at 2026-08-12 11:10:33 on dist-test-slave-drl0"
server_key: "0746c7f8c29388feb000ae686bb210d7"
server_key_iv: "cdaef28fd96be5c03724412ec07e4edb"
server_key_version: "encryptionkey@0"
I20260812 11:10:33.554498 1707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:10:33.580111 1707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:10:33.583720 1707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:10:33.583949 1707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:10:33.584721 1707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:10:33.585727 1707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:10:33.585800 1707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:33.585875 1707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:10:33.585911 1707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:10:33.598093 1707 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:39135
I20260812 11:10:33.598117 1906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:39135 every 8 connection(s)
I20260812 11:10:33.599653 1707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:10:33.602306 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 1707
I20260812 11:10:33.602459 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:10:33.602795 17587 external_mini_cluster.cc:1428] Setting key 2d6cedd2e8b9a2d49a2a844241983afd
Aug 12 11:10:33 dist-test-slave-drl0 krb5kdc[1114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533033, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:10:33.615816 1248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:33.601830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:56303 (local address 127.17.44.254:43515)
0812 11:10:33.602155 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:33.602160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:33.602901 (+ 741us) server_negotiation.cc:407] Connection header received
0812 11:10:33.604638 (+ 1737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:33.604642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:33.604700 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:33.604813 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:33.606161 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.606723 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.608020 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.608235 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.608353 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:33.611726 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:33.611729 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:33.611731 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:33.611754 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:33.613158 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:33.613754 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:33.613757 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:33.613759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:33.613839 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:33.614339 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:33.614343 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:33.614345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:33.614652 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:33.614779 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:33.615470 (+ 691us) server_negotiation.cc:299] Negotiation successful
0812 11:10:33.615637 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260812 11:10:33.616580 1913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:33.602178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:43515 (local address 127.17.44.195:56303)
0812 11:10:33.602740 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:33.602782 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:33.604429 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:33.605008 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:33.605015 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:33.605381 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:10:33.606000 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:33.606024 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.606856 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.606860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:33.607884 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:33.607894 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.608391 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.608394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:33.608585 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:33.609344 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:10:33.609360 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:10:33.611435 (+ 2075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:10:33.613303 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:33.613308 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:33.613320 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:33.613605 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:33.613967 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:10:33.613970 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:10:33.613972 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:10:33.614125 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:10:33.614827 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:10:33.614833 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:10:33.615167 (+ 334us) client_negotiation.cc:769] Sending connection context
0812 11:10:33.615344 (+ 177us) client_negotiation.cc:240] Negotiation successful
0812 11:10:33.615603 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":147,"threads_started":1}
I20260812 11:10:33.618533 1907 heartbeater.cc:347] Connected to a master server at 127.17.44.254:43515
I20260812 11:10:33.618822 1907 heartbeater.cc:464] Registering TS with master...
I20260812 11:10:33.619380 1907 heartbeater.cc:511] Master 127.17.44.254:43515 requested a full tablet report, sending...
I20260812 11:10:33.620564 1190 ts_manager.cc:194] Registered new tserver with Master: 9845eda5459e4861bbf9936ba040b245 (127.17.44.195:39135)
I20260812 11:10:33.621199 1190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.195@KRBTEST.COM'} at 127.17.44.195:56303
I20260812 11:10:33.631240 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:10:33.642578 1248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:33.633368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39874 (local address 127.17.44.254:43515)
0812 11:10:33.633535 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:33.633540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:33.633641 (+ 101us) server_negotiation.cc:407] Connection header received
0812 11:10:33.633737 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:33.633739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:33.633776 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:33.633850 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:10:33.634595 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.635145 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.636129 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.636418 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.636560 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:33.638582 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:10:33.638585 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:10:33.638587 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:10:33.638614 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:10:33.640219 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:33.640935 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:33.640940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:33.640981 (+ 41us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:33.641092 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:10:33.641577 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:10:33.641599 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:10:33.641602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:10:33.641990 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:10:33.642170 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:33.642352 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 11:10:33.642451 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 11:10:33.645908 1190 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39874:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:10:33.649415 1190 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:10:33.665968 1924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:33.658676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:39135 (local address 127.0.0.1:37870)
0812 11:10:33.659564 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:33.659600 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:33.659779 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:33.661086 (+ 1307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:33.661093 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:33.661119 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:33.661275 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:33.661282 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.663103 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.663107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:33.664277 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:33.664287 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.665391 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.665395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:33.665503 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:33.665548 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 11:10:33.665667 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 11:10:33.665787 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"mutex_wait_us":17,"thread_start_us":169,"threads_started":1}
I20260812 11:10:33.665992 1927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:33.658775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37870 (local address 127.17.44.195:39135)
0812 11:10:33.659314 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:33.659327 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:33.659704 (+ 377us) server_negotiation.cc:407] Connection header received
0812 11:10:33.659940 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:33.659947 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:33.660877 (+ 930us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:33.661449 (+ 572us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:33.661603 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.662692 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.664416 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.665247 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.665368 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:33.665515 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:33.665717 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 11:10:33.665839 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":171,"threads_started":1}
I20260812 11:10:33.666021 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:33.659874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:39117 (local address 127.0.0.1:40972)
0812 11:10:33.660483 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:33.660497 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:33.660675 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:33.661649 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:33.661651 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:33.661666 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:33.661736 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:33.661741 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.662737 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.662742 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:33.664203 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:33.664215 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.665604 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.665606 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:33.665730 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:33.665753 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 11:10:33.665827 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 11:10:33.665892 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"mutex_wait_us":5,"thread_start_us":120,"threads_started":1}
I20260812 11:10:33.666527 1926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:33.658676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:46725 (local address 127.0.0.1:55730)
0812 11:10:33.660206 (+ 1530us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:33.660221 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:33.660473 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:33.661125 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:33.661130 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:33.661153 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:33.661265 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:33.661272 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.662899 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.662902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:33.664909 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:33.664917 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.666054 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.666061 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:33.666159 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:33.666193 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 11:10:33.666298 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 11:10:33.666381 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1456,"mutex_wait_us":100,"thread_start_us":128,"threads_started":1}
I20260812 11:10:33.666563 1932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:33.660083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40972 (local address 127.17.44.193:39117)
0812 11:10:33.660651 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:33.660663 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:33.660690 (+ 27us) server_negotiation.cc:407] Connection header received
0812 11:10:33.661202 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:33.661208 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:33.661391 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:33.661524 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:33.661854 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.662608 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.664416 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.665247 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.665368 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:33.665515 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:33.666297 (+ 782us) server_negotiation.cc:299] Negotiation successful
0812 11:10:33.666394 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":130,"threads_started":1}
I20260812 11:10:33.667508 1930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:33.659062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55730 (local address 127.17.44.194:46725)
0812 11:10:33.659494 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:33.659504 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:33.660588 (+ 1084us) server_negotiation.cc:407] Connection header received
0812 11:10:33.660674 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:33.660680 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:33.660876 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:33.661064 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:33.661403 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.662657 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.665037 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.665764 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.665861 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:33.665986 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:33.667235 (+ 1249us) server_negotiation.cc:299] Negotiation successful
0812 11:10:33.667348 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":123,"threads_started":1}
I20260812 11:10:33.669278 1382 tablet_service.cc:1467] Processing CreateTablet for tablet b692f55ec96c410bb820fcd916493bc6 (DEFAULT_TABLE table=test-table [id=a5f7f33d99e84fdf812c2e8fd1551ccd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:10:33.669625 1800 tablet_service.cc:1467] Processing CreateTablet for tablet b692f55ec96c410bb820fcd916493bc6 (DEFAULT_TABLE table=test-table [id=a5f7f33d99e84fdf812c2e8fd1551ccd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:10:33.669843 1586 tablet_service.cc:1467] Processing CreateTablet for tablet b692f55ec96c410bb820fcd916493bc6 (DEFAULT_TABLE table=test-table [id=a5f7f33d99e84fdf812c2e8fd1551ccd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:10:33.670471 1382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b692f55ec96c410bb820fcd916493bc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:33.670601 1800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b692f55ec96c410bb820fcd916493bc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:33.670663 1586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b692f55ec96c410bb820fcd916493bc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:10:33.678363 1935 tablet_bootstrap.cc:492] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245: Bootstrap starting.
I20260812 11:10:33.679011 1938 tablet_bootstrap.cc:492] T b692f55ec96c410bb820fcd916493bc6 P 9f6e08d9e6c44711bfc88a5facf6fa5b: Bootstrap starting.
I20260812 11:10:33.679733 1937 tablet_bootstrap.cc:492] T b692f55ec96c410bb820fcd916493bc6 P 4e3307fc56314e39b1b88eecd76455d3: Bootstrap starting.
I20260812 11:10:33.681252 1938 tablet_bootstrap.cc:654] T b692f55ec96c410bb820fcd916493bc6 P 9f6e08d9e6c44711bfc88a5facf6fa5b: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:33.682200 1938 log.cc:824] T b692f55ec96c410bb820fcd916493bc6 P 9f6e08d9e6c44711bfc88a5facf6fa5b: Log is configured to *not* fsync() on all Append() calls
I20260812 11:10:33.682914 1935 tablet_bootstrap.cc:654] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:33.682924 1937 tablet_bootstrap.cc:654] T b692f55ec96c410bb820fcd916493bc6 P 4e3307fc56314e39b1b88eecd76455d3: Neither blocks nor log segments found. Creating new log.
I20260812 11:10:33.683913 1937 log.cc:824] T b692f55ec96c410bb820fcd916493bc6 P 4e3307fc56314e39b1b88eecd76455d3: Log is configured to *not* fsync() on all Append() calls
I20260812 11:10:33.683913 1935 log.cc:824] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245: Log is configured to *not* fsync() on all Append() calls
I20260812 11:10:33.684350 1938 tablet_bootstrap.cc:492] T b692f55ec96c410bb820fcd916493bc6 P 9f6e08d9e6c44711bfc88a5facf6fa5b: No bootstrap required, opened a new log
I20260812 11:10:33.684736 1938 ts_tablet_manager.cc:1397] T b692f55ec96c410bb820fcd916493bc6 P 9f6e08d9e6c44711bfc88a5facf6fa5b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 11:10:33.686240 1935 tablet_bootstrap.cc:492] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245: No bootstrap required, opened a new log
I20260812 11:10:33.686496 1935 ts_tablet_manager.cc:1397] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.006s
I20260812 11:10:33.686528 1937 tablet_bootstrap.cc:492] T b692f55ec96c410bb820fcd916493bc6 P 4e3307fc56314e39b1b88eecd76455d3: No bootstrap required, opened a new log
I20260812 11:10:33.686702 1937 ts_tablet_manager.cc:1397] T b692f55ec96c410bb820fcd916493bc6 P 4e3307fc56314e39b1b88eecd76455d3: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 11:10:33.690352 1938 raft_consensus.cc:348] T b692f55ec96c410bb820fcd916493bc6 P 9f6e08d9e6c44711bfc88a5facf6fa5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9845eda5459e4861bbf9936ba040b245" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 39135 } } peers { permanent_uuid: "4e3307fc56314e39b1b88eecd76455d3" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46725 } } peers { permanent_uuid: "9f6e08d9e6c44711bfc88a5facf6fa5b" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39117 } }
I20260812 11:10:33.690698 1938 raft_consensus.cc:374] T b692f55ec96c410bb820fcd916493bc6 P 9f6e08d9e6c44711bfc88a5facf6fa5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:33.690759 1938 raft_consensus.cc:729] T b692f55ec96c410bb820fcd916493bc6 P 9f6e08d9e6c44711bfc88a5facf6fa5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f6e08d9e6c44711bfc88a5facf6fa5b, State: Initialized, Role: FOLLOWER
I20260812 11:10:33.690785 1935 raft_consensus.cc:348] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9845eda5459e4861bbf9936ba040b245" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 39135 } } peers { permanent_uuid: "4e3307fc56314e39b1b88eecd76455d3" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46725 } } peers { permanent_uuid: "9f6e08d9e6c44711bfc88a5facf6fa5b" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39117 } }
I20260812 11:10:33.690785 1937 raft_consensus.cc:348] T b692f55ec96c410bb820fcd916493bc6 P 4e3307fc56314e39b1b88eecd76455d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9845eda5459e4861bbf9936ba040b245" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 39135 } } peers { permanent_uuid: "4e3307fc56314e39b1b88eecd76455d3" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46725 } } peers { permanent_uuid: "9f6e08d9e6c44711bfc88a5facf6fa5b" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39117 } }
I20260812 11:10:33.691102 1935 raft_consensus.cc:374] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:33.691102 1937 raft_consensus.cc:374] T b692f55ec96c410bb820fcd916493bc6 P 4e3307fc56314e39b1b88eecd76455d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:10:33.691169 1935 raft_consensus.cc:729] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9845eda5459e4861bbf9936ba040b245, State: Initialized, Role: FOLLOWER
I20260812 11:10:33.691169 1937 raft_consensus.cc:729] T b692f55ec96c410bb820fcd916493bc6 P 4e3307fc56314e39b1b88eecd76455d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e3307fc56314e39b1b88eecd76455d3, State: Initialized, Role: FOLLOWER
I20260812 11:10:33.691501 1938 consensus_queue.cc:260] T b692f55ec96c410bb820fcd916493bc6 P 9f6e08d9e6c44711bfc88a5facf6fa5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9845eda5459e4861bbf9936ba040b245" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 39135 } } peers { permanent_uuid: "4e3307fc56314e39b1b88eecd76455d3" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46725 } } peers { permanent_uuid: "9f6e08d9e6c44711bfc88a5facf6fa5b" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39117 } }
I20260812 11:10:33.691639 1937 consensus_queue.cc:260] T b692f55ec96c410bb820fcd916493bc6 P 4e3307fc56314e39b1b88eecd76455d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9845eda5459e4861bbf9936ba040b245" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 39135 } } peers { permanent_uuid: "4e3307fc56314e39b1b88eecd76455d3" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46725 } } peers { permanent_uuid: "9f6e08d9e6c44711bfc88a5facf6fa5b" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39117 } }
I20260812 11:10:33.691639 1935 consensus_queue.cc:260] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9845eda5459e4861bbf9936ba040b245" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 39135 } } peers { permanent_uuid: "4e3307fc56314e39b1b88eecd76455d3" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46725 } } peers { permanent_uuid: "9f6e08d9e6c44711bfc88a5facf6fa5b" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39117 } }
I20260812 11:10:33.692350 1482 heartbeater.cc:503] Master 127.17.44.254:43515 was elected leader, sending a full tablet report...
I20260812 11:10:33.692639 1938 ts_tablet_manager.cc:1428] T b692f55ec96c410bb820fcd916493bc6 P 9f6e08d9e6c44711bfc88a5facf6fa5b: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
I20260812 11:10:33.692639 1937 ts_tablet_manager.cc:1428] T b692f55ec96c410bb820fcd916493bc6 P 4e3307fc56314e39b1b88eecd76455d3: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260812 11:10:33.692639 1935 ts_tablet_manager.cc:1428] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 11:10:33.692941 1907 heartbeater.cc:503] Master 127.17.44.254:43515 was elected leader, sending a full tablet report...
I20260812 11:10:33.692945 1700 heartbeater.cc:503] Master 127.17.44.254:43515 was elected leader, sending a full tablet report...
I20260812 11:10:33.764175 1944 raft_consensus.cc:482] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:10:33.764391 1944 raft_consensus.cc:504] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9845eda5459e4861bbf9936ba040b245" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 39135 } } peers { permanent_uuid: "4e3307fc56314e39b1b88eecd76455d3" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46725 } } peers { permanent_uuid: "9f6e08d9e6c44711bfc88a5facf6fa5b" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39117 } }
I20260812 11:10:33.765700 1944 leader_election.cc:288] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e3307fc56314e39b1b88eecd76455d3 (127.17.44.194:46725), 9f6e08d9e6c44711bfc88a5facf6fa5b (127.17.44.193:39117)
I20260812 11:10:33.770138 1913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:33.766067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:46725 (local address 127.17.44.195:60817)
0812 11:10:33.766216 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:33.766233 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:33.766327 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:33.766786 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:33.766789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:33.766810 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:33.766990 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:33.766997 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.767889 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.767892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:33.768876 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:33.768886 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.769701 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.769705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:33.769813 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:33.769840 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 11:10:33.769931 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 11:10:33.770007 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 11:10:33.770200 1930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:33.766183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:60817 (local address 127.17.44.194:46725)
0812 11:10:33.766317 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:33.766322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:33.766339 (+ 17us) server_negotiation.cc:407] Connection header received
0812 11:10:33.766498 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:33.766504 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:33.766565 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:33.766639 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:33.767124 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.767768 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.769024 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.769482 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.769578 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:33.769620 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:33.769992 (+ 372us) server_negotiation.cc:299] Negotiation successful
0812 11:10:33.770072 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 11:10:33.770498 1932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:33.766564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:54869 (local address 127.17.44.193:39117)
0812 11:10:33.766673 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:33.766678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:33.766888 (+ 210us) server_negotiation.cc:407] Connection header received
0812 11:10:33.767014 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:33.767017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:33.767062 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:33.767146 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:10:33.767525 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.768123 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.769235 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.769816 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.769950 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:33.769994 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:33.770293 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 11:10:33.770383 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 11:10:33.770830 1948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:33.766351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:39117 (local address 127.17.44.195:54869)
0812 11:10:33.766789 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:33.766802 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:10:33.766888 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:10:33.767253 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:10:33.767256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:10:33.767273 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:10:33.767387 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:33.767392 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.768264 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.768269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:33.769081 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:10:33.769091 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.770006 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.770010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:10:33.770110 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:33.770134 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 11:10:33.770643 (+ 509us) client_negotiation.cc:240] Negotiation successful
0812 11:10:33.770726 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":155,"threads_started":1}
I20260812 11:10:33.770860 1626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b692f55ec96c410bb820fcd916493bc6" candidate_uuid: "9845eda5459e4861bbf9936ba040b245" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e3307fc56314e39b1b88eecd76455d3" is_pre_election: true
I20260812 11:10:33.771230 1626 raft_consensus.cc:2454] T b692f55ec96c410bb820fcd916493bc6 P 4e3307fc56314e39b1b88eecd76455d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9845eda5459e4861bbf9936ba040b245 in term 0.
I20260812 11:10:33.771497 1413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b692f55ec96c410bb820fcd916493bc6" candidate_uuid: "9845eda5459e4861bbf9936ba040b245" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f6e08d9e6c44711bfc88a5facf6fa5b" is_pre_election: true
I20260812 11:10:33.771759 1732 leader_election.cc:302] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e3307fc56314e39b1b88eecd76455d3, 9845eda5459e4861bbf9936ba040b245; no voters:
I20260812 11:10:33.771785 1413 raft_consensus.cc:2454] T b692f55ec96c410bb820fcd916493bc6 P 9f6e08d9e6c44711bfc88a5facf6fa5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9845eda5459e4861bbf9936ba040b245 in term 0.
I20260812 11:10:33.772050 1944 raft_consensus.cc:2781] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:10:33.772145 1944 raft_consensus.cc:482] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:10:33.772193 1944 raft_consensus.cc:3037] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:33.773439 1944 raft_consensus.cc:504] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9845eda5459e4861bbf9936ba040b245" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 39135 } } peers { permanent_uuid: "4e3307fc56314e39b1b88eecd76455d3" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46725 } } peers { permanent_uuid: "9f6e08d9e6c44711bfc88a5facf6fa5b" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39117 } }
I20260812 11:10:33.773924 1944 leader_election.cc:288] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245 [CANDIDATE]: Term 1 election: Requested vote from peers 4e3307fc56314e39b1b88eecd76455d3 (127.17.44.194:46725), 9f6e08d9e6c44711bfc88a5facf6fa5b (127.17.44.193:39117)
I20260812 11:10:33.774407 1413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b692f55ec96c410bb820fcd916493bc6" candidate_uuid: "9845eda5459e4861bbf9936ba040b245" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f6e08d9e6c44711bfc88a5facf6fa5b"
I20260812 11:10:33.774401 1626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b692f55ec96c410bb820fcd916493bc6" candidate_uuid: "9845eda5459e4861bbf9936ba040b245" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e3307fc56314e39b1b88eecd76455d3"
I20260812 11:10:33.774533 1626 raft_consensus.cc:3037] T b692f55ec96c410bb820fcd916493bc6 P 4e3307fc56314e39b1b88eecd76455d3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:33.774533 1413 raft_consensus.cc:3037] T b692f55ec96c410bb820fcd916493bc6 P 9f6e08d9e6c44711bfc88a5facf6fa5b [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:10:33.776090 1626 raft_consensus.cc:2454] T b692f55ec96c410bb820fcd916493bc6 P 4e3307fc56314e39b1b88eecd76455d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9845eda5459e4861bbf9936ba040b245 in term 1.
I20260812 11:10:33.776090 1413 raft_consensus.cc:2454] T b692f55ec96c410bb820fcd916493bc6 P 9f6e08d9e6c44711bfc88a5facf6fa5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9845eda5459e4861bbf9936ba040b245 in term 1.
I20260812 11:10:33.776438 1732 leader_election.cc:302] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e3307fc56314e39b1b88eecd76455d3, 9845eda5459e4861bbf9936ba040b245; no voters:
I20260812 11:10:33.776628 1944 raft_consensus.cc:2781] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:10:33.776854 1944 raft_consensus.cc:686] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245 [term 1 LEADER]: Becoming Leader. State: Replica: 9845eda5459e4861bbf9936ba040b245, State: Running, Role: LEADER
I20260812 11:10:33.777181 1944 consensus_queue.cc:237] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9845eda5459e4861bbf9936ba040b245" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 39135 } } peers { permanent_uuid: "4e3307fc56314e39b1b88eecd76455d3" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46725 } } peers { permanent_uuid: "9f6e08d9e6c44711bfc88a5facf6fa5b" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39117 } }
I20260812 11:10:33.780599 1190 catalog_manager.cc:5495] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9845eda5459e4861bbf9936ba040b245 (127.17.44.195). New cstate: current_term: 1 leader_uuid: "9845eda5459e4861bbf9936ba040b245" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9845eda5459e4861bbf9936ba040b245" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 39135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e3307fc56314e39b1b88eecd76455d3" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f6e08d9e6c44711bfc88a5facf6fa5b" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39117 } health_report { overall_health: UNKNOWN } } }
I20260812 11:10:33.821038 1927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:33.817275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37880 (local address 127.17.44.195:39135)
0812 11:10:33.817444 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 11:10:33.817450 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:10:33.817466 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:10:33.817561 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:10:33.817563 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:10:33.817619 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:10:33.817735 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:10:33.818000 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.818506 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.819502 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:10:33.819725 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:10:33.819844 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:10:33.820163 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:10:33.820584 (+ 421us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:10:33.820728 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 11:10:33.820773 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 11:10:33.820848 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 11:10:33.827384 1626 raft_consensus.cc:1261] T b692f55ec96c410bb820fcd916493bc6 P 4e3307fc56314e39b1b88eecd76455d3 [term 1 FOLLOWER]: Refusing update from remote peer 9845eda5459e4861bbf9936ba040b245: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:10:33.827385 1413 raft_consensus.cc:1261] T b692f55ec96c410bb820fcd916493bc6 P 9f6e08d9e6c44711bfc88a5facf6fa5b [term 1 FOLLOWER]: Refusing update from remote peer 9845eda5459e4861bbf9936ba040b245: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:10:33.828147 1944 consensus_queue.cc:1035] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e3307fc56314e39b1b88eecd76455d3" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:10:33.828406 1949 consensus_queue.cc:1035] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f6e08d9e6c44711bfc88a5facf6fa5b" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:10:33.839185 1958 mvcc.cc:205] Tried to move back new op lower bound from 7317639306549063680 to 7317639306354102272. Current Snapshot: MvccSnapshot[applied={T|T < 7317639306549063680}]
I20260812 11:10:33.847098 1190 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:10:33.847298 1190 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:10:33.849956 1190 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 33ca74abc79243c5bd53be832bebae7c: Sending DeleteTablet for 3 replicas of tablet b692f55ec96c410bb820fcd916493bc6
I20260812 11:10:33.850932 1800 tablet_service.cc:1514] Processing DeleteTablet for tablet b692f55ec96c410bb820fcd916493bc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:10:33 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:37870
I20260812 11:10:33.851127 1586 tablet_service.cc:1514] Processing DeleteTablet for tablet b692f55ec96c410bb820fcd916493bc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:10:33 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:55730
I20260812 11:10:33.851616 1382 tablet_service.cc:1514] Processing DeleteTablet for tablet b692f55ec96c410bb820fcd916493bc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:10:33 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:40972
I20260812 11:10:33.851689 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 1287
I20260812 11:10:33.854825 1966 tablet_replica.cc:330] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245: stopping tablet replica
I20260812 11:10:33.855134 1966 raft_consensus.cc:2229] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:10:33.855443 1966 raft_consensus.cc:2258] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:10:33.865777 1966 ts_tablet_manager.cc:1905] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:10:33.868834 1966 ts_tablet_manager.cc:1918] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:10:33.868950 1966 log.cc:1196] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/b692f55ec96c410bb820fcd916493bc6
I20260812 11:10:33.869289 1966 ts_tablet_manager.cc:1939] T b692f55ec96c410bb820fcd916493bc6 P 9845eda5459e4861bbf9936ba040b245: Deleting consensus metadata
I20260812 11:10:33.874161 1967 tablet_replica.cc:330] T b692f55ec96c410bb820fcd916493bc6 P 4e3307fc56314e39b1b88eecd76455d3: stopping tablet replica
I20260812 11:10:33.874514 1967 raft_consensus.cc:2229] T b692f55ec96c410bb820fcd916493bc6 P 4e3307fc56314e39b1b88eecd76455d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:10:33.874799 1967 raft_consensus.cc:2258] T b692f55ec96c410bb820fcd916493bc6 P 4e3307fc56314e39b1b88eecd76455d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:10:33.875303 1167 catalog_manager.cc:4841] TS 9845eda5459e4861bbf9936ba040b245 (127.17.44.195:39135): tablet b692f55ec96c410bb820fcd916493bc6 (table test-table [id=a5f7f33d99e84fdf812c2e8fd1551ccd]) successfully deleted
I20260812 11:10:33.876686 1967 ts_tablet_manager.cc:1905] T b692f55ec96c410bb820fcd916493bc6 P 4e3307fc56314e39b1b88eecd76455d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:10:33.879010 1967 ts_tablet_manager.cc:1918] T b692f55ec96c410bb820fcd916493bc6 P 4e3307fc56314e39b1b88eecd76455d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:10:33.879128 1967 log.cc:1196] T b692f55ec96c410bb820fcd916493bc6 P 4e3307fc56314e39b1b88eecd76455d3: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/b692f55ec96c410bb820fcd916493bc6
I20260812 11:10:33.879426 1967 ts_tablet_manager.cc:1939] T b692f55ec96c410bb820fcd916493bc6 P 4e3307fc56314e39b1b88eecd76455d3: Deleting consensus metadata
I20260812 11:10:33.880554 1168 catalog_manager.cc:4841] TS 4e3307fc56314e39b1b88eecd76455d3 (127.17.44.194:46725): tablet b692f55ec96c410bb820fcd916493bc6 (table test-table [id=a5f7f33d99e84fdf812c2e8fd1551ccd]) successfully deleted
W20260812 11:10:33.882920 1168 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.44.193:39117 (error 108)
I20260812 11:10:33.883054 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 1490
I20260812 11:10:33.889460 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:10:33.889154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:39117 (local address 127.0.0.1:40978)
0812 11:10:33.889289 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 11:10:33.889365 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.193:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 11:10:33.889644 1168 catalog_manager.cc:4584] TS 9f6e08d9e6c44711bfc88a5facf6fa5b (127.17.44.193:39117): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b692f55ec96c410bb820fcd916493bc6: Network error: Client connection negotiation failed: client connection to 127.17.44.193:39117: connect: Connection refused (error 111)
I20260812 11:10:33.895069 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 1707
I20260812 11:10:33.909915 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 1136
2026-08-12T11:10:33Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2803 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:10:33 dist-test-slave-drl0 krb5kdc[1976](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:10:33 dist-test-slave-drl0 krb5kdc[1976](info): set up 2 sockets
Aug 12 11:10:33 dist-test-slave-drl0 krb5kdc[1976](info): commencing operation
krb5kdc: starting...
W20260812 11:10:35.216202 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:10:35 dist-test-slave-drl0 krb5kdc[1976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:10:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:10:35Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.44.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.44.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.44.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/rangeradmin_127.17.44.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.44.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/rangeradmin_127.17.44.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.44.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.44.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.44.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/rangerlookup_127.17.44.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.44.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/rangerlookup_127.17.44.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.44.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.44.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.44.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/HTTP_127.17.44.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.44.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/HTTP_127.17.44.212@KRBTEST.COM.keytab.
I20260812 11:10:35.423424 17587 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskqOACJ0/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/postgres -l logfile start
2026-08-12 11:10:38.633 UTC [2024] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 11:10:38.634 UTC [2024] LOG: listening on IPv4 address "127.17.44.212", port 41935
2026-08-12 11:10:38.637 UTC [2024] LOG: listening on Unix socket "/tmp/.s.PGSQL.41935"
2026-08-12 11:10:38.679 UTC [2030] LOG: database system was shut down at 2026-08-12 11:10:36 UTC
2026-08-12 11:10:38.684 UTC [2024] LOG: database system is ready to accept connections
I20260812 11:10:39.942535 17587 mini_postgres.cc:96] Postgres bound to 41935
2026-08-12 11:10:39.952 UTC [2041] FATAL: database "slave" does not exist
127.17.44.212:41935 - accepting connections
I20260812 11:10:39.953178 17587 mini_ranger.cc:161] Starting Ranger...
I20260812 11:10:39.971933 17587 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 11:10:40.591439 17587 mini_ranger.cc:87] Created ranger Postgres database
I20260812 11:10:40.591554 17587 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ranger-admin
2026-08-12 11:10:40,961 [I] DB FLAVOR :POSTGRES
2026-08-12 11:10:40,962 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 11:10:40,962 [I] Checking connection..
2026-08-12 11:10:40,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:41935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 11:10:41,261 [I] Checking connection passed.
2026-08-12 11:10:41,262 [I] --------- Verifying version history table ---------
2026-08-12 11:10:41,262 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:41935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 11:10:41,561 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 11:10:41,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:41935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 11:10:41,857 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 11:10:41,857 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 11:10:41,857 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:41935/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 11:10:42.137 UTC [2133] WARNING: there is no transaction in progress
2026-08-12 11:10:42,148 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 11:10:42,148 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:41935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 11:10:42,444 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 11:10:42,444 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 11:10:42,445 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:41935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 11:10:42,730 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:41935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 11:10:43,014 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:41935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-drl0.c.gcp-upstream.internal','N') ;"
2026-08-12 11:10:43,293 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:41935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 11:10:43,557 [I] Table x_portal_user does not exist in database ranger
2026-08-12 11:10:43,558 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:41935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 11:10:43,847 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 11:10:43,847 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:41935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 11:10:44,141 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:41935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 11:10:44,442 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 11:10:44,442 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:41935/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 11:10:45.527 UTC [2300] WARNING: there is no transaction in progress
2026-08-12 11:10:45.546 UTC [2300] WARNING: there is no transaction in progress
2026-08-12 11:10:45.565 UTC [2300] WARNING: there is no transaction in progress
2026-08-12 11:10:45.584 UTC [2300] WARNING: there is no transaction in progress
2026-08-12 11:10:45.601 UTC [2300] WARNING: there is no transaction in progress
2026-08-12 11:10:45.621 UTC [2300] WARNING: there is no transaction in progress
2026-08-12 11:10:45.684 UTC [2300] WARNING: there is no transaction in progress
2026-08-12 11:10:45.691 UTC [2300] WARNING: there is no transaction in progress
2026-08-12 11:10:45.708 UTC [2300] WARNING: there is no transaction in progress
2026-08-12 11:10:45.720 UTC [2300] WARNING: there is no transaction in progress
2026-08-12 11:10:45.731 UTC [2300] WARNING: there is no transaction in progress
2026-08-12 11:10:45.745 UTC [2300] WARNING: there is no transaction in progress
2026-08-12 11:10:45.756 UTC [2300] WARNING: there is no transaction in progress
2026-08-12 11:10:45.764 UTC [2300] WARNING: there is no transaction in progress
2026-08-12 11:10:45.772 UTC [2300] WARNING: there is no transaction in progress
2026-08-12 11:10:45.780 UTC [2300] WARNING: there is no transaction in progress
2026-08-12 11:10:46,548 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 11:10:46,548 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:41935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 11:10:46,822 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 11:10:46,822 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:41935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 11:10:47,107 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 11:10:47,108 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:41935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 11:10:47,431 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 11:10:47,432 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:41935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 11:10:47,699 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:41935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 11:10:47,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:41935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-drl0.c.gcp-upstream.internal';"
2026-08-12 11:10:48,241 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 11:10:48,241 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:41935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 11:10:48,498 [I] DB_PATCHES have already been applied
I20260812 11:10:48.505039 17587 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ranger-admin:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskqOACJ0/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 11:10:48.505185 17587 mini_ranger.cc:193] Using host: 127.17.44.212
I20260812 11:10:48.509853 17587 mini_ranger.cc:227] Ranger admin URL: http://127.17.44.212:36721
Aug 12, 2026 11:10:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 11:10:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 11:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 11:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 11:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 11:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 11:10:49 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 11:10:49.934167 17587 mini_ranger.cc:160] Time spent starting Ranger: real 9.981s user 0.001s sys 0.013s
Aug 12, 2026 11:11:12 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 11:11:12 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 12, 2026 11:11:12 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 11:11:12 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 11:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 11:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 11:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 11:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 11:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 11:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 11:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 11:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 11:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 11:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 11:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 11:11:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 11:11:12 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 11:11:13 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 11:11:14.388345 17587 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.44.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.44.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.44.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/rangerkms_127.17.44.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.44.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/rangerkms_127.17.44.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.44.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.44.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.44.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/HTTP_127.17.44.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.44.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/HTTP_127.17.44.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 11:11:14 dist-test-slave-drl0 krb5kdc[1976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533074, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 11:11:14.505805 17587 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 11:11:14.522634 17587 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 11:11:15.528656 17587 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 11:11:15.528752 17587 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ranger-kms
I20260812 11:11:15.528864 17587 mini_ranger_kms.cc:227] Using postgres at 127.17.44.212:41935
2026-08-12 11:11:15,959 [I] DB FLAVOR :POSTGRES
2026-08-12 11:11:15,960 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 11:11:15,960 [I] Checking connection
2026-08-12 11:11:15,960 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:41935/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 11:11:16,269 [I] connection success
2026-08-12 11:11:16,270 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 11:11:16,270 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 11:11:16,270 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:41935/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 11:11:16,542 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 11:11:16,542 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 11:11:16,542 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 11:11:16,542 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:41935/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 11:11:16,849 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 11:11:17.422578 17587 mini_ranger_kms.cc:310] Created kms service
I20260812 11:11:17.615957 17587 mini_ranger_kms.cc:326] Created kudu user
I20260812 11:11:17.661449 17587 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 11:11:17 dist-test-slave-drl0 krb5kdc[1976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533074, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.44.212@KRBTEST.COM
I20260812 11:11:18.135090 17587 mini_ranger_kms.cc:384] Added ranger policy
I20260812 11:11:18.135326 17587 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ranger-kms:/tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 11:11:18.135352 17587 mini_ranger_kms.cc:240] Using host: 127.17.44.212
I20260812 11:11:18.141278 17587 mini_ranger_kms.cc:276] Ranger KMS PID: 2593
I20260812 11:11:18.141420 17587 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.44.212:42475
Aug 12, 2026 11:11:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 11:11:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 11:11:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 11:11:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 11:11:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 11:11:19 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 11:11:20 dist-test-slave-drl0 krb5kdc[1976](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786533080, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.44.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:11:20.923936 17587 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.418s user 0.018s sys 0.392s
I20260812 11:11:20.924093 17587 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 11:11:20.924225 17587 mini_ranger_kms.cc:401] 127.17.44.212:42475/kms/v1/keys
Aug 12 11:11:21 dist-test-slave-drl0 krb5kdc[1976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533080, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.44.212@KRBTEST.COM for HTTP/127.17.44.212@KRBTEST.COM
Aug 12 11:11:21 dist-test-slave-drl0 krb5kdc[1976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533080, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.44.212@KRBTEST.COM for HTTP/127.17.44.212@KRBTEST.COM
Aug 12 11:11:24 dist-test-slave-drl0 krb5kdc[1976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:11:24.234812 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:46317
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:39635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:46317
--ranger_config_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.44.212:42475/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:11:24.386188 2674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:11:24.386530 2674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:11:24.386575 2674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:11:24.391067 2674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:11:24.391353 2674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:11:24.391433 2674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:11:24.391522 2674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:11:24.391680 2674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:11:24.400341 2674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:39635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.44.212:42475/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:46317
--ranger_config_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:46317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:11:24.401497 2674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:11:24.402585 2674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:11:24.413101 2687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:11:24.413146 2681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:11:24.413357 2682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:11:24.413769 2674 server_base.cc:1034] running on GCE node
I20260812 11:11:24.414422 2674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:11:24.415621 2674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:11:24.416849 2674 hybrid_clock.cc:648] HybridClock initialized: now 1786533084416777 us; error 88 us; skew 500 ppm
Aug 12 11:11:24 dist-test-slave-drl0 krb5kdc[1976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533084, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:11:24.426224 2674 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:11:24.427328 2674 webserver.cc:460] Webserver started at http://127.17.44.254:45007/ using document root <none> and password file <none>
I20260812 11:11:24.427918 2674 fs_manager.cc:359] Metadata directory not provided
I20260812 11:11:24.427975 2674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:11:24.428205 2674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 11:11:24 dist-test-slave-drl0 krb5kdc[1976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533084, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for HTTP/127.17.44.212@KRBTEST.COM
I20260812 11:11:24.495536 2674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "4f69bcc9ab5341019e41e96ad953115e"
format_stamp: "Formatted at 2026-08-12 11:11:24 on dist-test-slave-drl0"
server_key: "b12f083b10a4ac53545fb1481748df74"
server_key_iv: "3a99a4b94d28f28d157102e4c35ee729"
server_key_version: "kuduclusterkey@0"
I20260812 11:11:24.496207 2674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "4f69bcc9ab5341019e41e96ad953115e"
format_stamp: "Formatted at 2026-08-12 11:11:24 on dist-test-slave-drl0"
server_key: "b12f083b10a4ac53545fb1481748df74"
server_key_iv: "3a99a4b94d28f28d157102e4c35ee729"
server_key_version: "kuduclusterkey@0"
I20260812 11:11:24.500494 2674 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 11:11:24.520126 2697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:11:24.521615 2674 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 11:11:24.521771 2674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "4f69bcc9ab5341019e41e96ad953115e"
format_stamp: "Formatted at 2026-08-12 11:11:24 on dist-test-slave-drl0"
server_key: "b12f083b10a4ac53545fb1481748df74"
server_key_iv: "3a99a4b94d28f28d157102e4c35ee729"
server_key_version: "kuduclusterkey@0"
I20260812 11:11:24.521904 2674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:11:24.533804 2674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:11:24.537938 2674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:11:24.539219 2674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:11:24.553653 2674 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:46317
I20260812 11:11:24.553687 2784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:46317 every 8 connection(s)
I20260812 11:11:24.555068 2674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:11:24.557330 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 2674
I20260812 11:11:24.557495 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:11:24.557816 17587 external_mini_cluster.cc:1428] Setting key 9b0522113a8e86797e759b623d62f55e
I20260812 11:11:24.560212 2785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:11:24.570611 2785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f69bcc9ab5341019e41e96ad953115e: Bootstrap starting.
Aug 12 11:11:24 dist-test-slave-drl0 krb5kdc[1976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:11:24.575366 2785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f69bcc9ab5341019e41e96ad953115e: Neither blocks nor log segments found. Creating new log.
I20260812 11:11:24.576570 2785 log.cc:824] T 00000000000000000000000000000000 P 4f69bcc9ab5341019e41e96ad953115e: Log is configured to *not* fsync() on all Append() calls
I20260812 11:11:24.579391 2785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f69bcc9ab5341019e41e96ad953115e: No bootstrap required, opened a new log
I20260812 11:11:24.581888 2788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:11:24.559686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38682 (local address 127.17.44.254:46317)
0812 11:11:24.560323 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0812 11:11:24.560351 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0812 11:11:24.563443 (+ 3092us) server_negotiation.cc:407] Connection header received
0812 11:11:24.564696 (+ 1253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:11:24.564721 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:11:24.565180 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:11:24.565544 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:11:24.566992 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:11:24.568882 (+ 1890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:11:24.570264 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:11:24.571124 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:11:24.573422 (+ 2298us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:11:24.574882 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:11:24.574890 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:11:24.574909 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:11:24.574969 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:11:24.577704 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:11:24.578255 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:11:24.578261 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:11:24.578268 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:11:24.578456 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:11:24.578832 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:11:24.578835 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:11:24.578837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:11:24.579409 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:11:24.580305 (+ 896us) server_negotiation.cc:1035] Waiting for connection context
0812 11:11:24.580457 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 11:11:24.580876 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":233,"threads_started":1}
I20260812 11:11:24.586480 2785 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f69bcc9ab5341019e41e96ad953115e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f69bcc9ab5341019e41e96ad953115e" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 46317 } }
I20260812 11:11:24.587015 2785 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f69bcc9ab5341019e41e96ad953115e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:11:24.587095 2785 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f69bcc9ab5341019e41e96ad953115e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f69bcc9ab5341019e41e96ad953115e, State: Initialized, Role: FOLLOWER
I20260812 11:11:24.587761 2785 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f69bcc9ab5341019e41e96ad953115e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f69bcc9ab5341019e41e96ad953115e" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 46317 } }
I20260812 11:11:24.587999 2785 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f69bcc9ab5341019e41e96ad953115e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:11:24.588075 2785 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f69bcc9ab5341019e41e96ad953115e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:11:24.588177 2785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f69bcc9ab5341019e41e96ad953115e [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:11:24.589564 2785 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f69bcc9ab5341019e41e96ad953115e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f69bcc9ab5341019e41e96ad953115e" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 46317 } }
I20260812 11:11:24.589985 2785 leader_election.cc:302] T 00000000000000000000000000000000 P 4f69bcc9ab5341019e41e96ad953115e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f69bcc9ab5341019e41e96ad953115e; no voters:
I20260812 11:11:24.590387 2785 leader_election.cc:288] T 00000000000000000000000000000000 P 4f69bcc9ab5341019e41e96ad953115e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:11:24.590533 2792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f69bcc9ab5341019e41e96ad953115e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:11:24.590775 2792 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f69bcc9ab5341019e41e96ad953115e [term 1 LEADER]: Becoming Leader. State: Replica: 4f69bcc9ab5341019e41e96ad953115e, State: Running, Role: LEADER
I20260812 11:11:24.591174 2792 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f69bcc9ab5341019e41e96ad953115e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f69bcc9ab5341019e41e96ad953115e" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 46317 } }
I20260812 11:11:24.591852 2785 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f69bcc9ab5341019e41e96ad953115e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:11:24.592617 2794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f69bcc9ab5341019e41e96ad953115e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f69bcc9ab5341019e41e96ad953115e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f69bcc9ab5341019e41e96ad953115e" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 46317 } } }
I20260812 11:11:24.592760 2794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f69bcc9ab5341019e41e96ad953115e [sys.catalog]: This master's current role is: LEADER
I20260812 11:11:24.592617 2792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f69bcc9ab5341019e41e96ad953115e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f69bcc9ab5341019e41e96ad953115e. Latest consensus state: current_term: 1 leader_uuid: "4f69bcc9ab5341019e41e96ad953115e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f69bcc9ab5341019e41e96ad953115e" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 46317 } } }
I20260812 11:11:24.592955 2792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f69bcc9ab5341019e41e96ad953115e [sys.catalog]: This master's current role is: LEADER
I20260812 11:11:24.597803 2785 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 11:11:24.599155 2807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:11:24.606242 2807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:11:24.637049 2807 catalog_manager.cc:1347] Generated new cluster ID: 3d968cec32924607b5c4b43dcb3895e3
I20260812 11:11:24.637130 2807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:11:24.655833 2807 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:11:24.658119 2807 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:11:24.672670 2807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f69bcc9ab5341019e41e96ad953115e: Generated new TSK 0
I20260812 11:11:24.673674 2807 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
I20260812 11:11:25.494884 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.44.212:42475/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:46317
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:39635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:11:25.641893 2854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:11:25.642233 2854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:11:25.642314 2854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:11:25.646737 2854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:11:25.646833 2854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:11:25.646998 2854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:11:25.653687 2854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:39635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.44.212:42475/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:46317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:11:25.654793 2854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:11:25.655898 2854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:11:25.666587 2864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:11:25.666860 2868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:11:25.666671 2865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:11:25.667294 2854 server_base.cc:1034] running on GCE node
I20260812 11:11:25.667725 2854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:11:25.668476 2854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:11:25.669737 2854 hybrid_clock.cc:648] HybridClock initialized: now 1786533085669666 us; error 91 us; skew 500 ppm
Aug 12 11:11:25 dist-test-slave-drl0 krb5kdc[1976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533085, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:11:25.673962 2854 init.cc:377] Logged in from keytab as kudu/127.17.44.193@KRBTEST.COM (short username kudu)
I20260812 11:11:25.675050 2854 webserver.cc:460] Webserver started at http://127.17.44.193:38173/ using document root <none> and password file <none>
I20260812 11:11:25.675670 2854 fs_manager.cc:359] Metadata directory not provided
I20260812 11:11:25.675776 2854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:11:25.676028 2854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 11:11:25 dist-test-slave-drl0 krb5kdc[1976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533085, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for HTTP/127.17.44.212@KRBTEST.COM
I20260812 11:11:25.694178 2854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "9398f83dc5024c3b82c92c57d38fc023"
format_stamp: "Formatted at 2026-08-12 11:11:25 on dist-test-slave-drl0"
server_key: "b99c07004685d681a03d28bca2b6c81c"
server_key_iv: "5d647eecb8ec50cd78123f771eac0b20"
server_key_version: "kuduclusterkey@0"
I20260812 11:11:25.694808 2854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "9398f83dc5024c3b82c92c57d38fc023"
format_stamp: "Formatted at 2026-08-12 11:11:25 on dist-test-slave-drl0"
server_key: "b99c07004685d681a03d28bca2b6c81c"
server_key_iv: "5d647eecb8ec50cd78123f771eac0b20"
server_key_version: "kuduclusterkey@0"
I20260812 11:11:25.699638 2854 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 11:11:25.722023 2878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:11:25.723754 2854 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:11:25.723914 2854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "9398f83dc5024c3b82c92c57d38fc023"
format_stamp: "Formatted at 2026-08-12 11:11:25 on dist-test-slave-drl0"
server_key: "b99c07004685d681a03d28bca2b6c81c"
server_key_iv: "5d647eecb8ec50cd78123f771eac0b20"
server_key_version: "kuduclusterkey@0"
I20260812 11:11:25.724066 2854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:11:25.756009 2854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:11:25.760269 2854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:11:25.760495 2854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:11:25.761230 2854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:11:25.762326 2854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:11:25.762384 2854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:11:25.762445 2854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:11:25.762476 2854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:11:25.779013 2854 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:33123
I20260812 11:11:25.780987 2854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:11:25.783203 3057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:33123 every 8 connection(s)
I20260812 11:11:25.784950 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 2854
I20260812 11:11:25.785081 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:11:25.785432 17587 external_mini_cluster.cc:1428] Setting key 93b62d2a6caffcab8a170296889ce236
Aug 12 11:11:25 dist-test-slave-drl0 krb5kdc[1976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533085, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.44.194@KRBTEST.COM; defaulting to no policy
I20260812 11:11:25.806313 3062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:11:25.785916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:41005 (local address 127.17.44.254:46317)
0812 11:11:25.792331 (+ 6415us) server_negotiation.cc:206] Beginning negotiation
0812 11:11:25.792335 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:11:25.792351 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:11:25.792394 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:11:25.792397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:11:25.792447 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:11:25.792585 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:11:25.794403 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:11:25.795311 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:11:25.797450 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:11:25.797811 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:11:25.797908 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:11:25.801368 (+ 3460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:11:25.801372 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:11:25.801375 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:11:25.801403 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:11:25.803702 (+ 2299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:11:25.804400 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:11:25.804404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:11:25.804406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:11:25.804506 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:11:25.804928 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:11:25.804931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:11:25.804946 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:11:25.805204 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:11:25.805309 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 11:11:25.805989 (+ 680us) server_negotiation.cc:299] Negotiation successful
0812 11:11:25.806118 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6319,"thread_start_us":66,"threads_started":1}
I20260812 11:11:25.807374 3061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:11:25.785739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:46317 (local address 127.17.44.193:41005)
0812 11:11:25.788591 (+ 2852us) negotiation.cc:107] Waiting for socket to connect
0812 11:11:25.788653 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 11:11:25.789890 (+ 1237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:11:25.792925 (+ 3035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:11:25.792947 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:11:25.793447 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:11:25.794196 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:11:25.794220 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:11:25.795552 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:11:25.795556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:11:25.797293 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:11:25.797307 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:11:25.798114 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:11:25.798118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:11:25.798343 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:11:25.799122 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:11:25.799152 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:11:25.801160 (+ 2008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:11:25.803857 (+ 2697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:11:25.803863 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:11:25.803880 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:11:25.804271 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:11:25.804622 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:11:25.804625 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:11:25.804627 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:11:25.804807 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:11:25.805382 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:11:25.805392 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:11:25.805728 (+ 336us) client_negotiation.cc:769] Sending connection context
0812 11:11:25.805954 (+ 226us) client_negotiation.cc:240] Negotiation successful
0812 11:11:25.806284 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2585,"thread_start_us":145,"threads_started":1}
I20260812 11:11:25.809434 3059 heartbeater.cc:347] Connected to a master server at 127.17.44.254:46317
I20260812 11:11:25.809684 3059 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.194@KRBTEST.COM" created.
I20260812 11:11:25.810271 3059 heartbeater.cc:511] Master 127.17.44.254:46317 requested a full tablet report, sending...
I20260812 11:11:25.812108 2717 ts_manager.cc:194] Registered new tserver with Master: 9398f83dc5024c3b82c92c57d38fc023 (127.17.44.193:33123)
I20260812 11:11:25.813936 2717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.193@KRBTEST.COM'} at 127.17.44.193:41005
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
I20260812 11:11:25.830763 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.44.212:42475/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:46317
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:39635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 11:11:25 dist-test-slave-drl0 krb5kdc[1976](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786533085, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 11:11:25.986248 3070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:11:25.986722 3070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:11:25.986860 3070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:11:25.992926 3070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:11:25.993240 3070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:11:25.993402 3070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:11:26.001180 3070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:39635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.44.212:42475/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:46317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:11:26.002490 3070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:11:26.003670 3070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:11:26.015561 3083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:11:26.016485 3081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:11:26.016693 3070 server_base.cc:1034] running on GCE node
W20260812 11:11:26.016996 3079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:11:26.017479 3070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:11:26.018150 3070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:11:26.019379 3070 hybrid_clock.cc:648] HybridClock initialized: now 1786533086019346 us; error 43 us; skew 500 ppm
Aug 12 11:11:26 dist-test-slave-drl0 krb5kdc[1976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533086, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:11:26.022565 3070 init.cc:377] Logged in from keytab as kudu/127.17.44.194@KRBTEST.COM (short username kudu)
I20260812 11:11:26.023636 3070 webserver.cc:460] Webserver started at http://127.17.44.194:45487/ using document root <none> and password file <none>
I20260812 11:11:26.024261 3070 fs_manager.cc:359] Metadata directory not provided
I20260812 11:11:26.024425 3070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:11:26.024722 3070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 11:11:26 dist-test-slave-drl0 krb5kdc[1976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533086, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for HTTP/127.17.44.212@KRBTEST.COM
I20260812 11:11:26.041680 3070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "c63cda800b41407094125794065663da"
format_stamp: "Formatted at 2026-08-12 11:11:26 on dist-test-slave-drl0"
server_key: "3499b24fb33a147ea27385df7b15d30f"
server_key_iv: "9daacff2f21be4de6a99987084b3f985"
server_key_version: "kuduclusterkey@0"
I20260812 11:11:26.042337 3070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "c63cda800b41407094125794065663da"
format_stamp: "Formatted at 2026-08-12 11:11:26 on dist-test-slave-drl0"
server_key: "3499b24fb33a147ea27385df7b15d30f"
server_key_iv: "9daacff2f21be4de6a99987084b3f985"
server_key_version: "kuduclusterkey@0"
I20260812 11:11:26.048413 3070 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260812 11:11:26.072484 3097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:11:26.074277 3070 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.000s
I20260812 11:11:26.074419 3070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "c63cda800b41407094125794065663da"
format_stamp: "Formatted at 2026-08-12 11:11:26 on dist-test-slave-drl0"
server_key: "3499b24fb33a147ea27385df7b15d30f"
server_key_iv: "9daacff2f21be4de6a99987084b3f985"
server_key_version: "kuduclusterkey@0"
I20260812 11:11:26.074568 3070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:11:26.100510 3070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:11:26.104732 3070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:11:26.105026 3070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:11:26.109858 3070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:11:26.111922 3070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:11:26.112133 3070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260812 11:11:26.112259 3070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:11:26.112323 3070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:11:26.142158 3070 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:44191
I20260812 11:11:26.143630 3070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:11:26.146626 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 3070
I20260812 11:11:26.146764 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:11:26.147181 17587 external_mini_cluster.cc:1428] Setting key 1eb3986599103e548859aff5513ff925
I20260812 11:11:26.145884 3274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:44191 every 8 connection(s)
Aug 12 11:11:26 dist-test-slave-drl0 krb5kdc[1976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533086, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:11:26.172688 3062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:11:26.148985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:36319 (local address 127.17.44.254:46317)
0812 11:11:26.149246 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0812 11:11:26.149252 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:11:26.152264 (+ 3012us) server_negotiation.cc:407] Connection header received
0812 11:11:26.153972 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:11:26.153977 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:11:26.154055 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:11:26.154124 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:11:26.156719 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.157600 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.159420 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.159791 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.159875 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:11:26.164930 (+ 5055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:11:26.164947 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:11:26.164950 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:11:26.164984 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:11:26.166814 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:11:26.169881 (+ 3067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:11:26.169888 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:11:26.169890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:11:26.170075 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:11:26.170741 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:11:26.170745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:11:26.170747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:11:26.171106 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:11:26.171218 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 11:11:26.172201 (+ 983us) server_negotiation.cc:299] Negotiation successful
0812 11:11:26.172431 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 11:11:26.175227 3279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:11:26.149205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:46317 (local address 127.17.44.194:36319)
0812 11:11:26.152017 (+ 2812us) negotiation.cc:107] Waiting for socket to connect
0812 11:11:26.152094 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0812 11:11:26.153615 (+ 1521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:11:26.154718 (+ 1103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:11:26.154730 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:11:26.155620 (+ 890us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:11:26.156456 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:11:26.156485 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.157801 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.157805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:11:26.159266 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:11:26.159278 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.160116 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.160120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:11:26.160617 (+ 497us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:11:26.162506 (+ 1889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:11:26.162531 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:11:26.164681 (+ 2150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:11:26.166991 (+ 2310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:11:26.166998 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:11:26.167019 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:11:26.167606 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:11:26.170264 (+ 2658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:11:26.170268 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:11:26.170270 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:11:26.170593 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:11:26.171307 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:11:26.171317 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:11:26.171821 (+ 504us) client_negotiation.cc:769] Sending connection context
0812 11:11:26.173268 (+ 1447us) client_negotiation.cc:240] Negotiation successful
0812 11:11:26.173987 (+ 719us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2356,"thread_start_us":381,"threads_started":1}
I20260812 11:11:26.181975 3277 heartbeater.cc:347] Connected to a master server at 127.17.44.254:46317
I20260812 11:11:26.182253 3277 heartbeater.cc:464] Registering TS with master...
I20260812 11:11:26.182997 3277 heartbeater.cc:511] Master 127.17.44.254:46317 requested a full tablet report, sending...
I20260812 11:11:26.185269 2717 ts_manager.cc:194] Registered new tserver with Master: c63cda800b41407094125794065663da (127.17.44.194:44191)
I20260812 11:11:26.186091 2717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.194@KRBTEST.COM'} at 127.17.44.194:36319
WARNING: no policy specified for kudu/127.17.44.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
I20260812 11:11:26.238533 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.44.212:42475/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:46317
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:39635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:11:26.440584 3286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:11:26.441072 3286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:11:26.441203 3286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:11:26.447718 3286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:11:26.447880 3286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:11:26.448040 3286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:11:26.456363 3286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:39635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.44.212:42475/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:46317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3286
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:11:26.458062 3286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:11:26.459573 3286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:11:26.476625 3292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:11:26.477264 3286 server_base.cc:1034] running on GCE node
W20260812 11:11:26.478534 3295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:11:26.483500 3291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:11:26.484191 3286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:11:26.485107 3286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:11:26.500582 3286 hybrid_clock.cc:648] HybridClock initialized: now 1786533086500408 us; error 185 us; skew 500 ppm
Aug 12 11:11:26 dist-test-slave-drl0 krb5kdc[1976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533086, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:11:26.504576 3286 init.cc:377] Logged in from keytab as kudu/127.17.44.195@KRBTEST.COM (short username kudu)
Aug 12 11:11:26 dist-test-slave-drl0 krb5kdc[1976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533085, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for HTTP/127.17.44.212@KRBTEST.COM
I20260812 11:11:26.506405 3286 webserver.cc:460] Webserver started at http://127.17.44.195:44703/ using document root <none> and password file <none>
I20260812 11:11:26.507189 3286 fs_manager.cc:359] Metadata directory not provided
I20260812 11:11:26.507282 3286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:11:26.507540 3286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 11:11:26 dist-test-slave-drl0 krb5kdc[1976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533086, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for HTTP/127.17.44.212@KRBTEST.COM
I20260812 11:11:26.535307 3286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "7b0f607401ce4b9fbcf1a42dc7ab71f1"
format_stamp: "Formatted at 2026-08-12 11:11:26 on dist-test-slave-drl0"
server_key: "38dff02528b982c5f0619423cadecf1c"
server_key_iv: "d4e406b805aee0faea1de64a57ce2a7b"
server_key_version: "kuduclusterkey@0"
I20260812 11:11:26.536062 3286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "7b0f607401ce4b9fbcf1a42dc7ab71f1"
format_stamp: "Formatted at 2026-08-12 11:11:26 on dist-test-slave-drl0"
server_key: "38dff02528b982c5f0619423cadecf1c"
server_key_iv: "d4e406b805aee0faea1de64a57ce2a7b"
server_key_version: "kuduclusterkey@0"
I20260812 11:11:26.544593 3286 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.000s
I20260812 11:11:26.592020 3302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:11:26.594650 3286 fs_manager.cc:725] Time spent opening block manager: real 0.034s user 0.002s sys 0.001s
I20260812 11:11:26.594806 3286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "7b0f607401ce4b9fbcf1a42dc7ab71f1"
format_stamp: "Formatted at 2026-08-12 11:11:26 on dist-test-slave-drl0"
server_key: "38dff02528b982c5f0619423cadecf1c"
server_key_iv: "d4e406b805aee0faea1de64a57ce2a7b"
server_key_version: "kuduclusterkey@0"
I20260812 11:11:26.594990 3286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:11:26.646443 3286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:11:26.650267 3286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:11:26.650485 3286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:11:26.651415 3286 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:11:26.652662 3286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:11:26.652729 3286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:11:26.652791 3286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:11:26.652824 3286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:11:26.667426 3286 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:34657
I20260812 11:11:26.668574 3286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:11:26.672955 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 3286
I20260812 11:11:26.673105 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:11:26.673431 3474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:34657 every 8 connection(s)
I20260812 11:11:26.673494 17587 external_mini_cluster.cc:1428] Setting key 12f5da0f0293a8efda4bbe09e0f4e536
Aug 12 11:11:26 dist-test-slave-drl0 krb5kdc[1976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533086, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:11:26.693053 3480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:11:26.676031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:46317 (local address 127.17.44.195:53135)
0812 11:11:26.677517 (+ 1486us) negotiation.cc:107] Waiting for socket to connect
0812 11:11:26.677573 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 11:11:26.678585 (+ 1012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:11:26.679477 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:11:26.679486 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:11:26.679855 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:11:26.680655 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:11:26.680676 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.681948 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.681952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:11:26.683483 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:11:26.683495 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.684180 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.684184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:11:26.684383 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:11:26.685111 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:11:26.685132 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:11:26.686908 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:11:26.689471 (+ 2563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:11:26.689478 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:11:26.689494 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:11:26.689842 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:11:26.690186 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:11:26.690189 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:11:26.690191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:11:26.690362 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:11:26.690847 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:11:26.690854 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:11:26.691468 (+ 614us) client_negotiation.cc:769] Sending connection context
0812 11:11:26.691666 (+ 198us) client_negotiation.cc:240] Negotiation successful
0812 11:11:26.692011 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1252,"thread_start_us":117,"threads_started":1}
I20260812 11:11:26.694095 3481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:11:26.677127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:53135 (local address 127.17.44.254:46317)
0812 11:11:26.679046 (+ 1919us) server_negotiation.cc:206] Beginning negotiation
0812 11:11:26.679051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:11:26.679065 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:11:26.679104 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:11:26.679107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:11:26.679158 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:11:26.680806 (+ 1648us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:11:26.680921 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.681786 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.683637 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.684003 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.687099 (+ 3096us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:11:26.687218 (+ 119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:11:26.687221 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:11:26.687224 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:11:26.687240 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:11:26.689202 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:11:26.689989 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:11:26.689993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:11:26.689995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:11:26.690082 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:11:26.690491 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:11:26.690495 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:11:26.690496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:11:26.690719 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:11:26.693711 (+ 2992us) server_negotiation.cc:1035] Waiting for connection context
0812 11:11:26.693794 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 11:11:26.693925 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1839,"thread_start_us":92,"threads_started":1}
I20260812 11:11:26.695200 3477 heartbeater.cc:347] Connected to a master server at 127.17.44.254:46317
I20260812 11:11:26.695433 3477 heartbeater.cc:464] Registering TS with master...
I20260812 11:11:26.696002 3477 heartbeater.cc:511] Master 127.17.44.254:46317 requested a full tablet report, sending...
I20260812 11:11:26.697115 2717 ts_manager.cc:194] Registered new tserver with Master: 7b0f607401ce4b9fbcf1a42dc7ab71f1 (127.17.44.195:34657)
I20260812 11:11:26.697870 2717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.195@KRBTEST.COM'} at 127.17.44.195:53135
I20260812 11:11:26.702916 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:11:26.718878 3481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:11:26.705370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38698 (local address 127.17.44.254:46317)
0812 11:11:26.705491 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 11:11:26.705495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:11:26.709511 (+ 4016us) server_negotiation.cc:407] Connection header received
0812 11:11:26.709692 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:11:26.709696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:11:26.709744 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:11:26.709792 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:11:26.710911 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.711738 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.712957 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.713304 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.713385 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:11:26.715288 (+ 1903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:11:26.715292 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:11:26.715295 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:11:26.715323 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:11:26.717056 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:11:26.717569 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:11:26.717573 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:11:26.717575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:11:26.717663 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:11:26.717977 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:11:26.717981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:11:26.717983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:11:26.718168 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:11:26.718319 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 11:11:26.718563 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 11:11:26.718689 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 11:11:26.721931 2717 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:11:26.725299 2717 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:11:26.752774 3502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:11:26.737349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:33123 (local address 127.0.0.1:34834)
0812 11:11:26.743919 (+ 6570us) negotiation.cc:107] Waiting for socket to connect
0812 11:11:26.743933 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:11:26.744029 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:11:26.744776 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:11:26.744779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:11:26.744800 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:11:26.744887 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:11:26.744892 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.746985 (+ 2093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.746989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:11:26.748347 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:11:26.748376 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.750549 (+ 2173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.750553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:11:26.752088 (+ 1535us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:11:26.752199 (+ 111us) client_negotiation.cc:769] Sending connection context
0812 11:11:26.752336 (+ 137us) client_negotiation.cc:240] Negotiation successful
0812 11:11:26.752601 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6511,"mutex_wait_us":4,"thread_start_us":113,"threads_started":1}
I20260812 11:11:26.753427 3510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:11:26.737780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34834 (local address 127.17.44.193:33123)
0812 11:11:26.738098 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0812 11:11:26.738106 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:11:26.744128 (+ 6022us) server_negotiation.cc:407] Connection header received
0812 11:11:26.744214 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:11:26.744222 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:11:26.744412 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:11:26.744544 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:11:26.745051 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.746116 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.748685 (+ 2569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.749975 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.750188 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:11:26.750446 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0812 11:11:26.753038 (+ 2592us) server_negotiation.cc:299] Negotiation successful
0812 11:11:26.753206 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":106,"threads_started":1}
I20260812 11:11:26.754048 3508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:11:26.736387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42102 (local address 127.17.44.195:34657)
0812 11:11:26.737555 (+ 1168us) server_negotiation.cc:206] Beginning negotiation
0812 11:11:26.737570 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:11:26.743136 (+ 5566us) server_negotiation.cc:407] Connection header received
0812 11:11:26.743210 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:11:26.743214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:11:26.743383 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:11:26.743541 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:11:26.744269 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.745601 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.750343 (+ 4742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.751430 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.751562 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:11:26.751803 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0812 11:11:26.753738 (+ 1935us) server_negotiation.cc:299] Negotiation successful
0812 11:11:26.753895 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1056,"thread_start_us":113,"threads_started":1}
I20260812 11:11:26.755110 3509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:11:26.735392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:44191 (local address 127.0.0.1:54710)
0812 11:11:26.737817 (+ 2425us) negotiation.cc:107] Waiting for socket to connect
0812 11:11:26.737852 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 11:11:26.737992 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:11:26.743722 (+ 5730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:11:26.743728 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:11:26.743768 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:11:26.743950 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:11:26.743962 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.748778 (+ 4816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.748781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:11:26.750102 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:11:26.750116 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.754585 (+ 4469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.754589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:11:26.754752 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:11:26.754781 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 11:11:26.754862 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 11:11:26.754969 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2250,"mutex_wait_us":2,"spinlock_wait_cycles":7680,"thread_start_us":192,"threads_started":1}
I20260812 11:11:26.755573 3503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:11:26.735521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54710 (local address 127.17.44.194:44191)
0812 11:11:26.742997 (+ 7476us) server_negotiation.cc:206] Beginning negotiation
0812 11:11:26.743007 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 11:11:26.743034 (+ 27us) server_negotiation.cc:407] Connection header received
0812 11:11:26.743125 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:11:26.743131 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:11:26.743374 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:11:26.743541 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:11:26.747235 (+ 3694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.748598 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.752239 (+ 3641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.754210 (+ 1971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.754352 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:11:26.754509 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 11:11:26.755277 (+ 768us) server_negotiation.cc:299] Negotiation successful
0812 11:11:26.755398 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7331,"thread_start_us":80,"threads_started":1}
I20260812 11:11:26.756372 3505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:11:26.735735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:34657 (local address 127.0.0.1:42102)
0812 11:11:26.742593 (+ 6858us) negotiation.cc:107] Waiting for socket to connect
0812 11:11:26.742608 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:11:26.742708 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:11:26.743722 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:11:26.743728 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:11:26.743767 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:11:26.744139 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:11:26.744146 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.745752 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.745755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:11:26.747091 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:11:26.747104 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.751910 (+ 4806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.751913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:11:26.752087 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:11:26.753398 (+ 1311us) client_negotiation.cc:769] Sending connection context
0812 11:11:26.756146 (+ 2748us) client_negotiation.cc:240] Negotiation successful
0812 11:11:26.756232 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6789,"mutex_wait_us":1520,"spinlock_wait_cycles":5888,"thread_start_us":136,"threads_started":1}
I20260812 11:11:26.756939 2949 tablet_service.cc:1467] Processing CreateTablet for tablet 1e9910137d624b9480454181b1ee67c3 (DEFAULT_TABLE table=test-table [id=77397f410de948dca22a3d7d3a503306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:11:26.758188 2949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e9910137d624b9480454181b1ee67c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:11:26.758273 3154 tablet_service.cc:1467] Processing CreateTablet for tablet 1e9910137d624b9480454181b1ee67c3 (DEFAULT_TABLE table=test-table [id=77397f410de948dca22a3d7d3a503306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:11:26.759335 3154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e9910137d624b9480454181b1ee67c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:11:26.759101 3355 tablet_service.cc:1467] Processing CreateTablet for tablet 1e9910137d624b9480454181b1ee67c3 (DEFAULT_TABLE table=test-table [id=77397f410de948dca22a3d7d3a503306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:11:26.759994 3355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e9910137d624b9480454181b1ee67c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:11:26.769033 3513 tablet_bootstrap.cc:492] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023: Bootstrap starting.
I20260812 11:11:26.772367 3513 tablet_bootstrap.cc:654] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023: Neither blocks nor log segments found. Creating new log.
I20260812 11:11:26.772379 3517 tablet_bootstrap.cc:492] T 1e9910137d624b9480454181b1ee67c3 P 7b0f607401ce4b9fbcf1a42dc7ab71f1: Bootstrap starting.
I20260812 11:11:26.773692 3513 log.cc:824] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023: Log is configured to *not* fsync() on all Append() calls
I20260812 11:11:26.776096 3517 tablet_bootstrap.cc:654] T 1e9910137d624b9480454181b1ee67c3 P 7b0f607401ce4b9fbcf1a42dc7ab71f1: Neither blocks nor log segments found. Creating new log.
I20260812 11:11:26.776477 3512 tablet_bootstrap.cc:492] T 1e9910137d624b9480454181b1ee67c3 P c63cda800b41407094125794065663da: Bootstrap starting.
I20260812 11:11:26.776837 3513 tablet_bootstrap.cc:492] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023: No bootstrap required, opened a new log
I20260812 11:11:26.777068 3513 ts_tablet_manager.cc:1397] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260812 11:11:26.777657 3517 log.cc:824] T 1e9910137d624b9480454181b1ee67c3 P 7b0f607401ce4b9fbcf1a42dc7ab71f1: Log is configured to *not* fsync() on all Append() calls
I20260812 11:11:26.779187 3512 tablet_bootstrap.cc:654] T 1e9910137d624b9480454181b1ee67c3 P c63cda800b41407094125794065663da: Neither blocks nor log segments found. Creating new log.
I20260812 11:11:26.781219 3517 tablet_bootstrap.cc:492] T 1e9910137d624b9480454181b1ee67c3 P 7b0f607401ce4b9fbcf1a42dc7ab71f1: No bootstrap required, opened a new log
I20260812 11:11:26.781414 3517 ts_tablet_manager.cc:1397] T 1e9910137d624b9480454181b1ee67c3 P 7b0f607401ce4b9fbcf1a42dc7ab71f1: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 11:11:26.782297 3512 log.cc:824] T 1e9910137d624b9480454181b1ee67c3 P c63cda800b41407094125794065663da: Log is configured to *not* fsync() on all Append() calls
I20260812 11:11:26.782542 3513 raft_consensus.cc:348] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c63cda800b41407094125794065663da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 44191 } } peers { permanent_uuid: "7b0f607401ce4b9fbcf1a42dc7ab71f1" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34657 } } peers { permanent_uuid: "9398f83dc5024c3b82c92c57d38fc023" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 33123 } }
I20260812 11:11:26.782917 3513 raft_consensus.cc:374] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:11:26.783035 3513 raft_consensus.cc:729] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9398f83dc5024c3b82c92c57d38fc023, State: Initialized, Role: FOLLOWER
I20260812 11:11:26.783639 3513 consensus_queue.cc:260] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c63cda800b41407094125794065663da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 44191 } } peers { permanent_uuid: "7b0f607401ce4b9fbcf1a42dc7ab71f1" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34657 } } peers { permanent_uuid: "9398f83dc5024c3b82c92c57d38fc023" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 33123 } }
I20260812 11:11:26.784839 3513 ts_tablet_manager.cc:1428] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260812 11:11:26.785239 3059 heartbeater.cc:503] Master 127.17.44.254:46317 was elected leader, sending a full tablet report...
I20260812 11:11:26.785892 3517 raft_consensus.cc:348] T 1e9910137d624b9480454181b1ee67c3 P 7b0f607401ce4b9fbcf1a42dc7ab71f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c63cda800b41407094125794065663da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 44191 } } peers { permanent_uuid: "7b0f607401ce4b9fbcf1a42dc7ab71f1" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34657 } } peers { permanent_uuid: "9398f83dc5024c3b82c92c57d38fc023" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 33123 } }
I20260812 11:11:26.786146 3517 raft_consensus.cc:374] T 1e9910137d624b9480454181b1ee67c3 P 7b0f607401ce4b9fbcf1a42dc7ab71f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:11:26.786211 3517 raft_consensus.cc:729] T 1e9910137d624b9480454181b1ee67c3 P 7b0f607401ce4b9fbcf1a42dc7ab71f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b0f607401ce4b9fbcf1a42dc7ab71f1, State: Initialized, Role: FOLLOWER
I20260812 11:11:26.786664 3517 consensus_queue.cc:260] T 1e9910137d624b9480454181b1ee67c3 P 7b0f607401ce4b9fbcf1a42dc7ab71f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c63cda800b41407094125794065663da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 44191 } } peers { permanent_uuid: "7b0f607401ce4b9fbcf1a42dc7ab71f1" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34657 } } peers { permanent_uuid: "9398f83dc5024c3b82c92c57d38fc023" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 33123 } }
I20260812 11:11:26.787657 3517 ts_tablet_manager.cc:1428] T 1e9910137d624b9480454181b1ee67c3 P 7b0f607401ce4b9fbcf1a42dc7ab71f1: Time spent starting tablet: real 0.006s user 0.007s sys 0.001s
I20260812 11:11:26.788007 3477 heartbeater.cc:503] Master 127.17.44.254:46317 was elected leader, sending a full tablet report...
W20260812 11:11:26.789152 3060 tablet.cc:2369] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:11:26.791777 3512 tablet_bootstrap.cc:492] T 1e9910137d624b9480454181b1ee67c3 P c63cda800b41407094125794065663da: No bootstrap required, opened a new log
I20260812 11:11:26.792050 3512 ts_tablet_manager.cc:1397] T 1e9910137d624b9480454181b1ee67c3 P c63cda800b41407094125794065663da: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.000s
I20260812 11:11:26.797111 3512 raft_consensus.cc:348] T 1e9910137d624b9480454181b1ee67c3 P c63cda800b41407094125794065663da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c63cda800b41407094125794065663da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 44191 } } peers { permanent_uuid: "7b0f607401ce4b9fbcf1a42dc7ab71f1" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34657 } } peers { permanent_uuid: "9398f83dc5024c3b82c92c57d38fc023" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 33123 } }
I20260812 11:11:26.797374 3512 raft_consensus.cc:374] T 1e9910137d624b9480454181b1ee67c3 P c63cda800b41407094125794065663da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:11:26.797456 3512 raft_consensus.cc:729] T 1e9910137d624b9480454181b1ee67c3 P c63cda800b41407094125794065663da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c63cda800b41407094125794065663da, State: Initialized, Role: FOLLOWER
I20260812 11:11:26.797907 3512 consensus_queue.cc:260] T 1e9910137d624b9480454181b1ee67c3 P c63cda800b41407094125794065663da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c63cda800b41407094125794065663da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 44191 } } peers { permanent_uuid: "7b0f607401ce4b9fbcf1a42dc7ab71f1" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34657 } } peers { permanent_uuid: "9398f83dc5024c3b82c92c57d38fc023" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 33123 } }
I20260812 11:11:26.798695 3277 heartbeater.cc:503] Master 127.17.44.254:46317 was elected leader, sending a full tablet report...
I20260812 11:11:26.799000 3512 ts_tablet_manager.cc:1428] T 1e9910137d624b9480454181b1ee67c3 P c63cda800b41407094125794065663da: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 11:11:26.813041 3522 raft_consensus.cc:482] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:11:26.813256 3522 raft_consensus.cc:504] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c63cda800b41407094125794065663da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 44191 } } peers { permanent_uuid: "7b0f607401ce4b9fbcf1a42dc7ab71f1" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34657 } } peers { permanent_uuid: "9398f83dc5024c3b82c92c57d38fc023" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 33123 } }
I20260812 11:11:26.814558 3522 leader_election.cc:288] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c63cda800b41407094125794065663da (127.17.44.194:44191), 7b0f607401ce4b9fbcf1a42dc7ab71f1 (127.17.44.195:34657)
I20260812 11:11:26.820642 3503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:11:26.815441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:47763 (local address 127.17.44.194:44191)
0812 11:11:26.815601 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 11:11:26.815605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:11:26.815938 (+ 333us) server_negotiation.cc:407] Connection header received
0812 11:11:26.816137 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:11:26.816141 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:11:26.816187 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:11:26.816233 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:11:26.816653 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.817417 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.819256 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.819891 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.819992 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:11:26.820053 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 11:11:26.820413 (+ 360us) server_negotiation.cc:299] Negotiation successful
0812 11:11:26.820506 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 11:11:26.821184 3528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:11:26.814827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:44191 (local address 127.17.44.193:47763)
0812 11:11:26.815846 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0812 11:11:26.815861 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:11:26.816019 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:11:26.816374 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:11:26.816377 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:11:26.816393 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:11:26.816535 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:11:26.816541 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.817590 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.817593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:11:26.819109 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:11:26.819121 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.820169 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.820173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:11:26.820265 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:11:26.820291 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 11:11:26.820869 (+ 578us) client_negotiation.cc:240] Negotiation successful
0812 11:11:26.821020 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":926,"mutex_wait_us":3,"thread_start_us":157,"threads_started":1}
I20260812 11:11:26.821394 3508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:11:26.815300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:59047 (local address 127.17.44.195:34657)
0812 11:11:26.815450 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 11:11:26.815455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:11:26.816025 (+ 570us) server_negotiation.cc:407] Connection header received
0812 11:11:26.816213 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:11:26.816216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:11:26.816267 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:11:26.816311 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:11:26.816706 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.817449 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.819965 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.820511 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.820598 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:11:26.820642 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 11:11:26.821200 (+ 558us) server_negotiation.cc:299] Negotiation successful
0812 11:11:26.821274 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 11:11:26.821880 3200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e9910137d624b9480454181b1ee67c3" candidate_uuid: "9398f83dc5024c3b82c92c57d38fc023" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c63cda800b41407094125794065663da" is_pre_election: true
I20260812 11:11:26.822160 3529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:11:26.814883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:34657 (local address 127.17.44.193:59047)
0812 11:11:26.815932 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0812 11:11:26.815947 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:11:26.816097 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:11:26.816447 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:11:26.816450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:11:26.816465 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:11:26.816591 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:11:26.816597 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.818543 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.818547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:11:26.819819 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:11:26.819831 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.820937 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.820940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:11:26.821050 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:11:26.821073 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 11:11:26.821847 (+ 774us) client_negotiation.cc:240] Negotiation successful
0812 11:11:26.821924 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":974,"mutex_wait_us":29,"thread_start_us":228,"threads_started":1}
I20260812 11:11:26.822243 3200 raft_consensus.cc:2454] T 1e9910137d624b9480454181b1ee67c3 P c63cda800b41407094125794065663da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9398f83dc5024c3b82c92c57d38fc023 in term 0.
I20260812 11:11:26.822781 3411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e9910137d624b9480454181b1ee67c3" candidate_uuid: "9398f83dc5024c3b82c92c57d38fc023" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b0f607401ce4b9fbcf1a42dc7ab71f1" is_pre_election: true
I20260812 11:11:26.822850 2882 leader_election.cc:302] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9398f83dc5024c3b82c92c57d38fc023, c63cda800b41407094125794065663da; no voters:
I20260812 11:11:26.823078 3411 raft_consensus.cc:2454] T 1e9910137d624b9480454181b1ee67c3 P 7b0f607401ce4b9fbcf1a42dc7ab71f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9398f83dc5024c3b82c92c57d38fc023 in term 0.
I20260812 11:11:26.823246 3522 raft_consensus.cc:2781] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:11:26.823331 3522 raft_consensus.cc:482] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:11:26.823374 3522 raft_consensus.cc:3037] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:11:26.825176 3522 raft_consensus.cc:504] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c63cda800b41407094125794065663da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 44191 } } peers { permanent_uuid: "7b0f607401ce4b9fbcf1a42dc7ab71f1" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34657 } } peers { permanent_uuid: "9398f83dc5024c3b82c92c57d38fc023" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 33123 } }
I20260812 11:11:26.825582 3522 leader_election.cc:288] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023 [CANDIDATE]: Term 1 election: Requested vote from peers c63cda800b41407094125794065663da (127.17.44.194:44191), 7b0f607401ce4b9fbcf1a42dc7ab71f1 (127.17.44.195:34657)
I20260812 11:11:26.826016 3411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e9910137d624b9480454181b1ee67c3" candidate_uuid: "9398f83dc5024c3b82c92c57d38fc023" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b0f607401ce4b9fbcf1a42dc7ab71f1"
I20260812 11:11:26.826064 3200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e9910137d624b9480454181b1ee67c3" candidate_uuid: "9398f83dc5024c3b82c92c57d38fc023" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c63cda800b41407094125794065663da"
I20260812 11:11:26.826133 3411 raft_consensus.cc:3037] T 1e9910137d624b9480454181b1ee67c3 P 7b0f607401ce4b9fbcf1a42dc7ab71f1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:11:26.826182 3200 raft_consensus.cc:3037] T 1e9910137d624b9480454181b1ee67c3 P c63cda800b41407094125794065663da [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:11:26.827629 3411 raft_consensus.cc:2454] T 1e9910137d624b9480454181b1ee67c3 P 7b0f607401ce4b9fbcf1a42dc7ab71f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9398f83dc5024c3b82c92c57d38fc023 in term 1.
I20260812 11:11:26.828022 3200 raft_consensus.cc:2454] T 1e9910137d624b9480454181b1ee67c3 P c63cda800b41407094125794065663da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9398f83dc5024c3b82c92c57d38fc023 in term 1.
I20260812 11:11:26.828096 2881 leader_election.cc:302] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b0f607401ce4b9fbcf1a42dc7ab71f1, 9398f83dc5024c3b82c92c57d38fc023; no voters:
I20260812 11:11:26.828300 3522 raft_consensus.cc:2781] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:11:26.828514 3522 raft_consensus.cc:686] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023 [term 1 LEADER]: Becoming Leader. State: Replica: 9398f83dc5024c3b82c92c57d38fc023, State: Running, Role: LEADER
I20260812 11:11:26.828868 3522 consensus_queue.cc:237] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c63cda800b41407094125794065663da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 44191 } } peers { permanent_uuid: "7b0f607401ce4b9fbcf1a42dc7ab71f1" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34657 } } peers { permanent_uuid: "9398f83dc5024c3b82c92c57d38fc023" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 33123 } }
I20260812 11:11:26.834414 2715 catalog_manager.cc:5495] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9398f83dc5024c3b82c92c57d38fc023 (127.17.44.193). New cstate: current_term: 1 leader_uuid: "9398f83dc5024c3b82c92c57d38fc023" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c63cda800b41407094125794065663da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 44191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b0f607401ce4b9fbcf1a42dc7ab71f1" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9398f83dc5024c3b82c92c57d38fc023" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 33123 } health_report { overall_health: HEALTHY } } }
I20260812 11:11:26.872948 3510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:11:26.867718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34838 (local address 127.17.44.193:33123)
0812 11:11:26.867842 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 11:11:26.867847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:11:26.867860 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:11:26.867944 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:11:26.867947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:11:26.868015 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:11:26.868114 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:11:26.868490 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.869236 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.870995 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:11:26.871410 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:11:26.871559 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:11:26.871937 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:11:26.872394 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:11:26.872536 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 11:11:26.872598 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 11:11:26.872659 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 11:11:26.880820 3411 raft_consensus.cc:1261] T 1e9910137d624b9480454181b1ee67c3 P 7b0f607401ce4b9fbcf1a42dc7ab71f1 [term 1 FOLLOWER]: Refusing update from remote peer 9398f83dc5024c3b82c92c57d38fc023: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:11:26.880813 3200 raft_consensus.cc:1261] T 1e9910137d624b9480454181b1ee67c3 P c63cda800b41407094125794065663da [term 1 FOLLOWER]: Refusing update from remote peer 9398f83dc5024c3b82c92c57d38fc023: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:11:26.881632 3531 consensus_queue.cc:1035] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b0f607401ce4b9fbcf1a42dc7ab71f1" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 34657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:11:26.882081 3531 consensus_queue.cc:1035] T 1e9910137d624b9480454181b1ee67c3 P 9398f83dc5024c3b82c92c57d38fc023 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c63cda800b41407094125794065663da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 44191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:11:26.897609 3539 mvcc.cc:205] Tried to move back new op lower bound from 7317639523855360000 to 7317639523653808128. Current Snapshot: MvccSnapshot[applied={T|T < 7317639523855360000}]
I20260812 11:11:26.897823 3552 mvcc.cc:205] Tried to move back new op lower bound from 7317639523855360000 to 7317639523653808128. Current Snapshot: MvccSnapshot[applied={T|T < 7317639523855360000}]
Aug 12 11:11:26 dist-test-slave-drl0 krb5kdc[1976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533085, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for HTTP/127.17.44.212@KRBTEST.COM
W20260812 11:11:26.902127 3278 tablet.cc:2369] T 1e9910137d624b9480454181b1ee67c3 P c63cda800b41407094125794065663da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:11:26.906988 3541 mvcc.cc:205] Tried to move back new op lower bound from 7317639523855360000 to 7317639523653808128. Current Snapshot: MvccSnapshot[applied={T|T < 7317639523855360000}]
I20260812 11:11:26.921916 2715 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:11:26.922111 2715 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:11:26.925290 2715 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f69bcc9ab5341019e41e96ad953115e: Sending DeleteTablet for 3 replicas of tablet 1e9910137d624b9480454181b1ee67c3
I20260812 11:11:26.926357 3154 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e9910137d624b9480454181b1ee67c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:11:26 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:54710
I20260812 11:11:26.927122 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 2854
I20260812 11:11:26.947261 3565 tablet_replica.cc:330] T 1e9910137d624b9480454181b1ee67c3 P c63cda800b41407094125794065663da: stopping tablet replica
I20260812 11:11:26.947983 3565 raft_consensus.cc:2229] T 1e9910137d624b9480454181b1ee67c3 P c63cda800b41407094125794065663da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:11:26.948700 3565 raft_consensus.cc:2258] T 1e9910137d624b9480454181b1ee67c3 P c63cda800b41407094125794065663da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:11:26.951061 3565 ts_tablet_manager.cc:1905] T 1e9910137d624b9480454181b1ee67c3 P c63cda800b41407094125794065663da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:11:26.954980 3565 ts_tablet_manager.cc:1918] T 1e9910137d624b9480454181b1ee67c3 P c63cda800b41407094125794065663da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:11:26.955229 3355 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e9910137d624b9480454181b1ee67c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:11:26 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:42102
I20260812 11:11:26.956908 3567 tablet_replica.cc:330] T 1e9910137d624b9480454181b1ee67c3 P 7b0f607401ce4b9fbcf1a42dc7ab71f1: stopping tablet replica
I20260812 11:11:26.957331 3567 raft_consensus.cc:2229] T 1e9910137d624b9480454181b1ee67c3 P 7b0f607401ce4b9fbcf1a42dc7ab71f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:11:26.957522 3565 log.cc:1196] T 1e9910137d624b9480454181b1ee67c3 P c63cda800b41407094125794065663da: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/1e9910137d624b9480454181b1ee67c3
I20260812 11:11:26.957731 3567 raft_consensus.cc:2258] T 1e9910137d624b9480454181b1ee67c3 P 7b0f607401ce4b9fbcf1a42dc7ab71f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:11:26.960319 3567 ts_tablet_manager.cc:1905] T 1e9910137d624b9480454181b1ee67c3 P 7b0f607401ce4b9fbcf1a42dc7ab71f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:11:26.960636 3565 ts_tablet_manager.cc:1939] T 1e9910137d624b9480454181b1ee67c3 P c63cda800b41407094125794065663da: Deleting consensus metadata
I20260812 11:11:26.963176 3567 ts_tablet_manager.cc:1918] T 1e9910137d624b9480454181b1ee67c3 P 7b0f607401ce4b9fbcf1a42dc7ab71f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:11:26.963202 2701 catalog_manager.cc:4841] TS c63cda800b41407094125794065663da (127.17.44.194:44191): tablet 1e9910137d624b9480454181b1ee67c3 (table test-table [id=77397f410de948dca22a3d7d3a503306]) successfully deleted
I20260812 11:11:26.963279 3567 log.cc:1196] T 1e9910137d624b9480454181b1ee67c3 P 7b0f607401ce4b9fbcf1a42dc7ab71f1: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/1e9910137d624b9480454181b1ee67c3
I20260812 11:11:26.963614 3567 ts_tablet_manager.cc:1939] T 1e9910137d624b9480454181b1ee67c3 P 7b0f607401ce4b9fbcf1a42dc7ab71f1: Deleting consensus metadata
I20260812 11:11:26.964706 2700 catalog_manager.cc:4841] TS 7b0f607401ce4b9fbcf1a42dc7ab71f1 (127.17.44.195:34657): tablet 1e9910137d624b9480454181b1ee67c3 (table test-table [id=77397f410de948dca22a3d7d3a503306]) successfully deleted
W20260812 11:11:26.970319 2698 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.44.193:33123 (error 108)
I20260812 11:11:26.970492 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 3070
I20260812 11:11:26.983363 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 3286
I20260812 11:11:26.995966 3505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:11:26.984293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:33123 (local address 127.0.0.1:34854)
0812 11:11:26.995614 (+ 11321us) negotiation.cc:107] Waiting for socket to connect
0812 11:11:26.995784 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.193:33123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11094}
W20260812 11:11:26.996356 2698 catalog_manager.cc:4584] TS 9398f83dc5024c3b82c92c57d38fc023 (127.17.44.193:33123): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1e9910137d624b9480454181b1ee67c3: Network error: Client connection negotiation failed: client connection to 127.17.44.193:33123: connect: Connection refused (error 111)
I20260812 11:11:27.006215 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 2674
I20260812 11:11:27.023962 17587 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 11:11:27.372622 17587 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 11:11:27.372870 17587 mini_ranger.cc:66] Stopping Ranger...
I20260812 11:11:27.728612 17587 mini_ranger.cc:68] Stopped Ranger
2026-08-12 11:11:27.729 UTC [2024] LOG: received smart shutdown request
2026-08-12 11:11:27.733 UTC [2024] LOG: background worker "logical replication launcher" (PID 2038) exited with exit code 1
2026-08-12 11:11:27.733 UTC [2031] LOG: shutting down
2026-08-12 11:11:27.919 UTC [2024] LOG: database system is shut down
2026-08-12T11:11:27Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (54137 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:11:28 dist-test-slave-drl0 krb5kdc[3579](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:11:28 dist-test-slave-drl0 krb5kdc[3579](info): set up 2 sockets
Aug 12 11:11:28 dist-test-slave-drl0 krb5kdc[3579](info): commencing operation
krb5kdc: starting...
W20260812 11:11:29.353585 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:11:29 dist-test-slave-drl0 krb5kdc[3579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:11:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:11:29Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.44.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.44.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.44.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/rangeradmin_127.17.44.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.44.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/rangeradmin_127.17.44.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.44.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.44.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.44.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/rangerlookup_127.17.44.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.44.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/rangerlookup_127.17.44.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.44.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.44.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.44.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/HTTP_127.17.44.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.44.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/HTTP_127.17.44.212@KRBTEST.COM.keytab.
I20260812 11:11:29.558669 17587 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskqOACJ0/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/postgres -l logfile start
2026-08-12 11:11:32.413 UTC [3622] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 11:11:32.414 UTC [3622] LOG: listening on IPv4 address "127.17.44.212", port 46721
2026-08-12 11:11:32.418 UTC [3622] LOG: listening on Unix socket "/tmp/.s.PGSQL.46721"
2026-08-12 11:11:32.443 UTC [3626] LOG: database system was shut down at 2026-08-12 11:11:30 UTC
2026-08-12 11:11:32.448 UTC [3622] LOG: database system is ready to accept connections
I20260812 11:11:33.646476 17587 mini_postgres.cc:96] Postgres bound to 46721
2026-08-12 11:11:33.654 UTC [3637] FATAL: database "slave" does not exist
127.17.44.212:46721 - accepting connections
I20260812 11:11:33.654685 17587 mini_ranger.cc:161] Starting Ranger...
I20260812 11:11:33.671108 17587 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 11:11:34.230685 17587 mini_ranger.cc:87] Created ranger Postgres database
I20260812 11:11:34.230777 17587 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ranger-admin
2026-08-12 11:11:34,543 [I] DB FLAVOR :POSTGRES
2026-08-12 11:11:34,544 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 11:11:34,544 [I] Checking connection..
2026-08-12 11:11:34,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:46721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 11:11:34,835 [I] Checking connection passed.
2026-08-12 11:11:34,835 [I] --------- Verifying version history table ---------
2026-08-12 11:11:34,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:46721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 11:11:35,109 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 11:11:35,110 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:46721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 11:11:35,369 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 11:11:35,370 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 11:11:35,370 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:46721/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 11:11:35.621 UTC [3742] WARNING: there is no transaction in progress
2026-08-12 11:11:35,631 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 11:11:35,631 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:46721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 11:11:35,897 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 11:11:35,897 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 11:11:35,897 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:46721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 11:11:36,163 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:46721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 11:11:36,448 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:46721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-drl0.c.gcp-upstream.internal','N') ;"
2026-08-12 11:11:36,702 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:46721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 11:11:36,997 [I] Table x_portal_user does not exist in database ranger
2026-08-12 11:11:36,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:46721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 11:11:37,268 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 11:11:37,269 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:46721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 11:11:37,552 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:46721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 11:11:37,820 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 11:11:37,820 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:46721/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 11:11:38.895 UTC [3914] WARNING: there is no transaction in progress
2026-08-12 11:11:38.912 UTC [3914] WARNING: there is no transaction in progress
2026-08-12 11:11:38.933 UTC [3914] WARNING: there is no transaction in progress
2026-08-12 11:11:38.950 UTC [3914] WARNING: there is no transaction in progress
2026-08-12 11:11:38.971 UTC [3914] WARNING: there is no transaction in progress
2026-08-12 11:11:38.993 UTC [3914] WARNING: there is no transaction in progress
2026-08-12 11:11:39.059 UTC [3914] WARNING: there is no transaction in progress
2026-08-12 11:11:39.068 UTC [3914] WARNING: there is no transaction in progress
2026-08-12 11:11:39.087 UTC [3914] WARNING: there is no transaction in progress
2026-08-12 11:11:39.099 UTC [3914] WARNING: there is no transaction in progress
2026-08-12 11:11:39.112 UTC [3914] WARNING: there is no transaction in progress
2026-08-12 11:11:39.128 UTC [3914] WARNING: there is no transaction in progress
2026-08-12 11:11:39.139 UTC [3914] WARNING: there is no transaction in progress
2026-08-12 11:11:39.146 UTC [3914] WARNING: there is no transaction in progress
2026-08-12 11:11:39.154 UTC [3914] WARNING: there is no transaction in progress
2026-08-12 11:11:39.163 UTC [3914] WARNING: there is no transaction in progress
2026-08-12 11:11:39,972 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 11:11:39,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:46721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 11:11:40,239 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 11:11:40,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:46721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 11:11:40,543 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 11:11:40,543 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:46721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 11:11:40,843 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 11:11:40,843 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:46721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 11:11:41,123 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:46721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 11:11:41,422 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:46721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-drl0.c.gcp-upstream.internal';"
2026-08-12 11:11:41,709 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 11:11:41,710 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:46721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 11:11:41,993 [I] DB_PATCHES have already been applied
I20260812 11:11:41.999320 17587 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ranger-admin:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskqOACJ0/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 11:11:41.999419 17587 mini_ranger.cc:193] Using host: 127.17.44.212
I20260812 11:11:42.005482 17587 mini_ranger.cc:227] Ranger admin URL: http://127.17.44.212:47135
Aug 12, 2026 11:11:42 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 11:11:42 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 11:11:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 11:11:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 11:11:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 11:11:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 11:11:42 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 11:11:43.449560 17587 mini_ranger.cc:160] Time spent starting Ranger: real 9.795s user 0.005s sys 0.007s
Aug 12, 2026 11:12:04 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 11:12:04 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 12, 2026 11:12:04 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 11:12:05 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 11:12:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 11:12:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 11:12:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 11:12:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 11:12:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 11:12:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 11:12:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 11:12:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 11:12:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 11:12:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 11:12:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 11:12:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 11:12:05 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 11:12:06 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 11:12:06.628726 17587 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.44.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.44.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.44.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/rangerkms_127.17.44.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.44.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/rangerkms_127.17.44.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.44.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.44.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.44.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/HTTP_127.17.44.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.44.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/HTTP_127.17.44.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 11:12:06 dist-test-slave-drl0 krb5kdc[3579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533126, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 11:12:06.741128 17587 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 11:12:06.753886 17587 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 11:12:07.750669 17587 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 11:12:07.750771 17587 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ranger-kms
I20260812 11:12:07.750794 17587 mini_ranger_kms.cc:227] Using postgres at 127.17.44.212:46721
2026-08-12 11:12:07,963 [I] DB FLAVOR :POSTGRES
2026-08-12 11:12:07,964 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 11:12:07,964 [I] Checking connection
2026-08-12 11:12:07,964 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:46721/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 11:12:08,238 [I] connection success
2026-08-12 11:12:08,238 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 11:12:08,238 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 11:12:08,238 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:46721/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 11:12:08,527 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 11:12:08,528 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 11:12:08,528 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 11:12:08,529 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.44.212:46721/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 11:12:08,857 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 11:12:09.392020 17587 mini_ranger_kms.cc:310] Created kms service
I20260812 11:12:09.579761 17587 mini_ranger_kms.cc:326] Created kudu user
I20260812 11:12:09.637377 17587 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 11:12:09 dist-test-slave-drl0 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533126, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.44.212@KRBTEST.COM
I20260812 11:12:10.106958 17587 mini_ranger_kms.cc:384] Added ranger policy
I20260812 11:12:10.107123 17587 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ranger-kms:/tmp/dist-test-taskqOACJ0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskqOACJ0/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 11:12:10.107146 17587 mini_ranger_kms.cc:240] Using host: 127.17.44.212
I20260812 11:12:10.112167 17587 mini_ranger_kms.cc:276] Ranger KMS PID: 4207
I20260812 11:12:10.112264 17587 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.44.212:39715
Aug 12, 2026 11:12:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 11:12:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 11:12:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 11:12:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 11:12:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 11:12:11 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 11:12:12 dist-test-slave-drl0 krb5kdc[3579](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786533132, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.44.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:12.881371 17587 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.140s user 0.023s sys 0.173s
I20260812 11:12:12.881487 17587 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 11:12:12.881577 17587 mini_ranger_kms.cc:401] 127.17.44.212:39715/kms/v1/keys
Aug 12 11:12:13 dist-test-slave-drl0 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533132, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.44.212@KRBTEST.COM for HTTP/127.17.44.212@KRBTEST.COM
Aug 12 11:12:13 dist-test-slave-drl0 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533132, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.44.212@KRBTEST.COM for HTTP/127.17.44.212@KRBTEST.COM
Aug 12 11:12:15 dist-test-slave-drl0 krb5kdc[3579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:12:15.895051 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:45555
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:45493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:45555
--ranger_config_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.44.212:39715/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:12:16.021317 4301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:16.021585 4301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:16.021626 4301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:16.025825 4301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:12:16.025887 4301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:16.025909 4301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:16.025929 4301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:12:16.025986 4301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:12:16.031600 4301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:45493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.44.212:39715/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:45555
--ranger_config_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:45555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:16.032718 4301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:16.033766 4301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:16.043268 4308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:16.043298 4306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:16.043505 4301 server_base.cc:1034] running on GCE node
W20260812 11:12:16.043579 4310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:16.044293 4301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:16.045704 4301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:16.046900 4301 hybrid_clock.cc:648] HybridClock initialized: now 1786533136046865 us; error 51 us; skew 500 ppm
Aug 12 11:12:16 dist-test-slave-drl0 krb5kdc[3579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533136, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:16.050757 4301 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:12:16.052145 4301 webserver.cc:460] Webserver started at http://127.17.44.254:38871/ using document root <none> and password file <none>
I20260812 11:12:16.052958 4301 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:16.053025 4301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:12:16.053418 4301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 11:12:16 dist-test-slave-drl0 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533136, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for HTTP/127.17.44.212@KRBTEST.COM
I20260812 11:12:16.103452 4301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "fcb846f0b6bc4560b181b6c0f574cf13"
format_stamp: "Formatted at 2026-08-12 11:12:16 on dist-test-slave-drl0"
server_key: "963d6e069fb4fa71845884117fb74881"
server_key_iv: "c56a7f403fa032d006f98cfd9b9c3a3e"
server_key_version: "kuduclusterkey@0"
I20260812 11:12:16.104183 4301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "fcb846f0b6bc4560b181b6c0f574cf13"
format_stamp: "Formatted at 2026-08-12 11:12:16 on dist-test-slave-drl0"
server_key: "963d6e069fb4fa71845884117fb74881"
server_key_iv: "c56a7f403fa032d006f98cfd9b9c3a3e"
server_key_version: "kuduclusterkey@0"
I20260812 11:12:16.108234 4301 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260812 11:12:16.124701 4325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:16.126187 4301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 11:12:16.126335 4301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "fcb846f0b6bc4560b181b6c0f574cf13"
format_stamp: "Formatted at 2026-08-12 11:12:16 on dist-test-slave-drl0"
server_key: "963d6e069fb4fa71845884117fb74881"
server_key_iv: "c56a7f403fa032d006f98cfd9b9c3a3e"
server_key_version: "kuduclusterkey@0"
I20260812 11:12:16.126470 4301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:16.146234 4301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:16.149276 4301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:16.149493 4301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:16.158269 4301 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:45555
I20260812 11:12:16.158476 4396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:45555 every 8 connection(s)
I20260812 11:12:16.159447 4301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:12:16.162330 4399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:12:16.164744 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 4301
I20260812 11:12:16.164889 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:12:16.165190 17587 external_mini_cluster.cc:1428] Setting key bc17442cb59ed05bae72ae3b559d62ab
I20260812 11:12:16.170307 4399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13: Bootstrap starting.
I20260812 11:12:16.173913 4399 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13: Neither blocks nor log segments found. Creating new log.
Aug 12 11:12:16 dist-test-slave-drl0 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:12:16.174818 4399 log.cc:824] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13: Log is configured to *not* fsync() on all Append() calls
I20260812 11:12:16.177345 4399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13: No bootstrap required, opened a new log
I20260812 11:12:16.180235 4406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:16.166602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38572 (local address 127.17.44.254:45555)
0812 11:12:16.167034 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:16.167065 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:16.167102 (+ 37us) server_negotiation.cc:407] Connection header received
0812 11:12:16.167748 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:16.167766 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:16.168078 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:16.168418 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:16.169232 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:16.170230 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:16.171241 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:16.171709 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:16.171893 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:16.174723 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:16.174729 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:16.174742 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:16.174777 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:16.177225 (+ 2448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:16.177776 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:16.177781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:16.177786 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:16.177891 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:16.178196 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:16.178199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:16.178201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:16.178590 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:16.178971 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:16.179228 (+ 257us) server_negotiation.cc:299] Negotiation successful
0812 11:12:16.179526 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":138,"threads_started":1}
I20260812 11:12:16.181525 4399 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb846f0b6bc4560b181b6c0f574cf13" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 45555 } }
I20260812 11:12:16.181800 4399 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:12:16.181882 4399 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb846f0b6bc4560b181b6c0f574cf13, State: Initialized, Role: FOLLOWER
I20260812 11:12:16.182341 4399 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb846f0b6bc4560b181b6c0f574cf13" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 45555 } }
I20260812 11:12:16.182523 4399 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:12:16.182588 4399 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:12:16.182679 4399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:12:16.183763 4399 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb846f0b6bc4560b181b6c0f574cf13" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 45555 } }
I20260812 11:12:16.184121 4399 leader_election.cc:302] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcb846f0b6bc4560b181b6c0f574cf13; no voters:
I20260812 11:12:16.184445 4399 leader_election.cc:288] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:12:16.184641 4410 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:12:16.184923 4410 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [term 1 LEADER]: Becoming Leader. State: Replica: fcb846f0b6bc4560b181b6c0f574cf13, State: Running, Role: LEADER
I20260812 11:12:16.185281 4410 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb846f0b6bc4560b181b6c0f574cf13" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 45555 } }
I20260812 11:12:16.185716 4399 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:12:16.186622 4410 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcb846f0b6bc4560b181b6c0f574cf13. Latest consensus state: current_term: 1 leader_uuid: "fcb846f0b6bc4560b181b6c0f574cf13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb846f0b6bc4560b181b6c0f574cf13" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 45555 } } }
I20260812 11:12:16.186838 4410 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [sys.catalog]: This master's current role is: LEADER
I20260812 11:12:16.187211 4411 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcb846f0b6bc4560b181b6c0f574cf13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb846f0b6bc4560b181b6c0f574cf13" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 45555 } } }
I20260812 11:12:16.187363 4411 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [sys.catalog]: This master's current role is: LEADER
I20260812 11:12:16.187368 4425 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:12:16.190722 4425 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:12:16.191231 4399 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 11:12:16.201452 4425 catalog_manager.cc:1347] Generated new cluster ID: 01d5f806d82c46fd8f935c6cfb7bc3e2
I20260812 11:12:16.201614 4425 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:12:16.218777 4425 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:12:16.220209 4425 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:12:16.228971 4425 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13: Generated new TSK 0
I20260812 11:12:16.229868 4425 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
I20260812 11:12:17.050316 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.44.212:39715/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:45555
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:45493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:12:17.190164 4479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:17.190593 4479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:17.190696 4479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:17.197137 4479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:17.197233 4479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:17.197325 4479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:12:17.205417 4479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:45493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.44.212:39715/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:45555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:17.206985 4479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:17.208441 4479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:17.218811 4491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:17.219094 4493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:17.218778 4490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:17.220242 4479 server_base.cc:1034] running on GCE node
I20260812 11:12:17.220691 4479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:17.221474 4479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:17.222668 4479 hybrid_clock.cc:648] HybridClock initialized: now 1786533137222623 us; error 59 us; skew 500 ppm
Aug 12 11:12:17 dist-test-slave-drl0 krb5kdc[3579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533137, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:17.225641 4479 init.cc:377] Logged in from keytab as kudu/127.17.44.193@KRBTEST.COM (short username kudu)
I20260812 11:12:17.226572 4479 webserver.cc:460] Webserver started at http://127.17.44.193:39959/ using document root <none> and password file <none>
I20260812 11:12:17.227309 4479 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:17.227407 4479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:12:17.227625 4479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 11:12:17 dist-test-slave-drl0 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533137, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for HTTP/127.17.44.212@KRBTEST.COM
I20260812 11:12:17.247402 4479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "a3d664ea3f5a4edd9d08173e78cd12d1"
format_stamp: "Formatted at 2026-08-12 11:12:17 on dist-test-slave-drl0"
server_key: "a11da8d76cf34202b2665358b848b8b9"
server_key_iv: "ca14db2487518fdae2e57d254ca308e2"
server_key_version: "kuduclusterkey@0"
I20260812 11:12:17.248320 4479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "a3d664ea3f5a4edd9d08173e78cd12d1"
format_stamp: "Formatted at 2026-08-12 11:12:17 on dist-test-slave-drl0"
server_key: "a11da8d76cf34202b2665358b848b8b9"
server_key_iv: "ca14db2487518fdae2e57d254ca308e2"
server_key_version: "kuduclusterkey@0"
I20260812 11:12:17.254164 4479 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 11:12:17.271647 4501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:17.273489 4479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:12:17.273643 4479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "a3d664ea3f5a4edd9d08173e78cd12d1"
format_stamp: "Formatted at 2026-08-12 11:12:17 on dist-test-slave-drl0"
server_key: "a11da8d76cf34202b2665358b848b8b9"
server_key_iv: "ca14db2487518fdae2e57d254ca308e2"
server_key_version: "kuduclusterkey@0"
I20260812 11:12:17.273797 4479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:17.290808 4479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:17.294610 4479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:17.295048 4479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:17.296160 4479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:12:17.297271 4479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:12:17.297387 4479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:17.297605 4479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:12:17.297688 4479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:17.309104 4479 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:36947
I20260812 11:12:17.310338 4479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:12:17.312750 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 4479
I20260812 11:12:17.312881 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:12:17.313191 17587 external_mini_cluster.cc:1428] Setting key 8b3782fd46d96828984c797292629293
I20260812 11:12:17.330423 4674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:36947 every 8 connection(s)
WARNING: no policy specified for kudu/127.17.44.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.194@KRBTEST.COM" created.
Aug 12 11:12:17 dist-test-slave-drl0 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533137, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:12:17.362670 4679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:17.333171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:44755 (local address 127.17.44.254:45555)
0812 11:12:17.335704 (+ 2533us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:17.335711 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:17.336334 (+ 623us) server_negotiation.cc:407] Connection header received
0812 11:12:17.338346 (+ 2012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:17.338351 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:17.338430 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:17.338569 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:17.342389 (+ 3820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:17.343296 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:17.350723 (+ 7427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:17.351139 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:17.351239 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:17.357748 (+ 6509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:17.357753 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:17.357756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:17.357788 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:17.359947 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:17.360680 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:17.360684 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:17.360686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:17.360785 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:17.361290 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:17.361294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:17.361296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:17.361507 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:17.361644 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:17.362321 (+ 677us) server_negotiation.cc:299] Negotiation successful
0812 11:12:17.362462 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2428,"thread_start_us":92,"threads_started":1}
I20260812 11:12:17.363791 4681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:17.333751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:45555 (local address 127.17.44.193:44755)
0812 11:12:17.336036 (+ 2285us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:17.336100 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:17.337548 (+ 1448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:17.338997 (+ 1449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:17.339011 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:17.339431 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:17.342166 (+ 2735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:17.342193 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:17.348475 (+ 6282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:17.348479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:17.350543 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:17.350557 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:17.351483 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:17.351488 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:17.351699 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:17.353643 (+ 1944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:17.353666 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:17.355611 (+ 1945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:17.360116 (+ 4505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:17.360123 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:17.360145 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:17.360543 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:17.360907 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:17.360910 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:17.360913 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:17.361166 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:17.361675 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:17.361685 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:17.362004 (+ 319us) client_negotiation.cc:769] Sending connection context
0812 11:12:17.362214 (+ 210us) client_negotiation.cc:240] Negotiation successful
0812 11:12:17.362599 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2027,"thread_start_us":161,"threads_started":1}
I20260812 11:12:17.365420 4675 heartbeater.cc:347] Connected to a master server at 127.17.44.254:45555
I20260812 11:12:17.365679 4675 heartbeater.cc:464] Registering TS with master...
I20260812 11:12:17.366308 4675 heartbeater.cc:511] Master 127.17.44.254:45555 requested a full tablet report, sending...
I20260812 11:12:17.368160 4351 ts_manager.cc:194] Registered new tserver with Master: a3d664ea3f5a4edd9d08173e78cd12d1 (127.17.44.193:36947)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
I20260812 11:12:17.369758 4351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.193@KRBTEST.COM'} at 127.17.44.193:44755
I20260812 11:12:17.370537 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.44.212:39715/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:45555
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:45493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 11:12:17 dist-test-slave-drl0 krb5kdc[3579](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786533137, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 11:12:17.510910 4684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:17.511257 4684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:17.511389 4684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:17.517256 4684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:17.517376 4684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:17.517493 4684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:12:17.524748 4684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:45493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.44.212:39715/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:45555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:17.525938 4684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:17.527108 4684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:17.538048 4707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:17.538693 4704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:17.540253 4701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:17.540436 4684 server_base.cc:1034] running on GCE node
I20260812 11:12:17.541123 4684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:17.541973 4684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:17.545300 4684 hybrid_clock.cc:648] HybridClock initialized: now 1786533137545266 us; error 52 us; skew 500 ppm
Aug 12 11:12:17 dist-test-slave-drl0 krb5kdc[3579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533137, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:17.548959 4684 init.cc:377] Logged in from keytab as kudu/127.17.44.194@KRBTEST.COM (short username kudu)
I20260812 11:12:17.550028 4684 webserver.cc:460] Webserver started at http://127.17.44.194:42519/ using document root <none> and password file <none>
I20260812 11:12:17.550653 4684 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:17.550717 4684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:12:17.550997 4684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 11:12:17 dist-test-slave-drl0 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533137, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for HTTP/127.17.44.212@KRBTEST.COM
I20260812 11:12:17.568468 4684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "f0fe682fba2f48a8bec6d66bf8ce4e42"
format_stamp: "Formatted at 2026-08-12 11:12:17 on dist-test-slave-drl0"
server_key: "b52de418e946d2b7b8cd457f9da90f86"
server_key_iv: "29b361125d82e8e73a4bb6d027838af7"
server_key_version: "kuduclusterkey@0"
I20260812 11:12:17.569324 4684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "f0fe682fba2f48a8bec6d66bf8ce4e42"
format_stamp: "Formatted at 2026-08-12 11:12:17 on dist-test-slave-drl0"
server_key: "b52de418e946d2b7b8cd457f9da90f86"
server_key_iv: "29b361125d82e8e73a4bb6d027838af7"
server_key_version: "kuduclusterkey@0"
I20260812 11:12:17.575281 4684 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 11:12:17.600858 4720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:17.602667 4684 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 11:12:17.602919 4684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "f0fe682fba2f48a8bec6d66bf8ce4e42"
format_stamp: "Formatted at 2026-08-12 11:12:17 on dist-test-slave-drl0"
server_key: "b52de418e946d2b7b8cd457f9da90f86"
server_key_iv: "29b361125d82e8e73a4bb6d027838af7"
server_key_version: "kuduclusterkey@0"
I20260812 11:12:17.603228 4684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:17.629070 4684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:17.633562 4684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:17.633845 4684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:17.634593 4684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:12:17.635888 4684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:12:17.636006 4684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:17.636109 4684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:12:17.636174 4684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:17.648626 4684 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:36029
I20260812 11:12:17.650162 4684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:12:17.651654 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 4684
I20260812 11:12:17.651791 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:12:17.652124 17587 external_mini_cluster.cc:1428] Setting key 9f07ce32c36cf89d92e76f55b78325ac
I20260812 11:12:17.651366 4903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:36029 every 8 connection(s)
Aug 12 11:12:17 dist-test-slave-drl0 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533137, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.44.195@KRBTEST.COM; defaulting to no policy
I20260812 11:12:17.682425 4679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:17.661022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:44485 (local address 127.17.44.254:45555)
0812 11:12:17.661151 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:17.661156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:17.663877 (+ 2721us) server_negotiation.cc:407] Connection header received
0812 11:12:17.665186 (+ 1309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:17.665189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:17.665245 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:17.665298 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:17.667022 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:17.667832 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:17.669576 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:17.669946 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:17.670032 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:17.676877 (+ 6845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:17.676882 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:17.676885 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:17.676918 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:17.679139 (+ 2221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:17.680212 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:17.680218 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:17.680220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:17.680335 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:17.680839 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:17.680843 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:17.680845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:17.681135 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:17.681253 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:17.682045 (+ 792us) server_negotiation.cc:299] Negotiation successful
0812 11:12:17.682221 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 11:12:17.684598 4907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:17.659872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:45555 (local address 127.17.44.194:44485)
0812 11:12:17.663690 (+ 3818us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:17.663756 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:17.664924 (+ 1168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:17.665712 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:17.665720 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:17.666098 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:17.666800 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:17.666820 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:17.668013 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:17.668016 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:17.669419 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:17.669432 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:17.670255 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:17.670260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:17.670464 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:17.674178 (+ 3714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:17.674202 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:17.676587 (+ 2385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:17.679433 (+ 2846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:17.679444 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:17.679474 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:17.680009 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:17.680492 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:17.680495 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:17.680498 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:17.680699 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:17.681339 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:17.681347 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:17.681714 (+ 367us) client_negotiation.cc:769] Sending connection context
0812 11:12:17.683032 (+ 1318us) client_negotiation.cc:240] Negotiation successful
0812 11:12:17.683466 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3358,"thread_start_us":147,"threads_started":1}
I20260812 11:12:17.687058 4904 heartbeater.cc:347] Connected to a master server at 127.17.44.254:45555
I20260812 11:12:17.687335 4904 heartbeater.cc:464] Registering TS with master...
I20260812 11:12:17.687973 4904 heartbeater.cc:511] Master 127.17.44.254:45555 requested a full tablet report, sending...
I20260812 11:12:17.689292 4351 ts_manager.cc:194] Registered new tserver with Master: f0fe682fba2f48a8bec6d66bf8ce4e42 (127.17.44.194:36029)
I20260812 11:12:17.690060 4351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.194@KRBTEST.COM'} at 127.17.44.194:44485
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
I20260812 11:12:17.718413 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.44.212:39715/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:45555
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:45493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:12:17.921906 4910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:17.922905 4910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:17.923081 4910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:17.929441 4910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:17.929566 4910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:17.929709 4910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:12:17.937491 4910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:45493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.44.212:39715/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:45555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:17.939529 4910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:17.941468 4910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:17.955929 4919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:17.956252 4910 server_base.cc:1034] running on GCE node
W20260812 11:12:17.956805 4925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:17.956012 4921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:17.957520 4910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:17.958415 4910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:17.965790 4910 hybrid_clock.cc:648] HybridClock initialized: now 1786533137965707 us; error 93 us; skew 500 ppm
Aug 12 11:12:17 dist-test-slave-drl0 krb5kdc[3579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533137, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:17.969691 4910 init.cc:377] Logged in from keytab as kudu/127.17.44.195@KRBTEST.COM (short username kudu)
I20260812 11:12:17.971179 4910 webserver.cc:460] Webserver started at http://127.17.44.195:44683/ using document root <none> and password file <none>
I20260812 11:12:17.971967 4910 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:17.972124 4910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:12:17.972456 4910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 11:12:17 dist-test-slave-drl0 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533137, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for HTTP/127.17.44.212@KRBTEST.COM
I20260812 11:12:17.993045 4910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "73a22a18776b48a7aefac3121d4949ec"
format_stamp: "Formatted at 2026-08-12 11:12:17 on dist-test-slave-drl0"
server_key: "710ddf962f7c25498a55200cd7e7546c"
server_key_iv: "2c89e5b7ef527c0e0a1dd7c931caab52"
server_key_version: "kuduclusterkey@0"
I20260812 11:12:17.995014 4910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "73a22a18776b48a7aefac3121d4949ec"
format_stamp: "Formatted at 2026-08-12 11:12:17 on dist-test-slave-drl0"
server_key: "710ddf962f7c25498a55200cd7e7546c"
server_key_iv: "2c89e5b7ef527c0e0a1dd7c931caab52"
server_key_version: "kuduclusterkey@0"
I20260812 11:12:18.010751 4910 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.003s sys 0.000s
I20260812 11:12:18.029695 4935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
Aug 12 11:12:18 dist-test-slave-drl0 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533137, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for HTTP/127.17.44.212@KRBTEST.COM
I20260812 11:12:18.032543 4910 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260812 11:12:18.032740 4910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "73a22a18776b48a7aefac3121d4949ec"
format_stamp: "Formatted at 2026-08-12 11:12:17 on dist-test-slave-drl0"
server_key: "710ddf962f7c25498a55200cd7e7546c"
server_key_iv: "2c89e5b7ef527c0e0a1dd7c931caab52"
server_key_version: "kuduclusterkey@0"
I20260812 11:12:18.032963 4910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:18.066123 4910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:18.069931 4910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:18.070168 4910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:18.070902 4910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:12:18.072033 4910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:12:18.072115 4910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:18.072207 4910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:12:18.072256 4910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:18.084070 4910 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:44755
I20260812 11:12:18.085268 4910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:12:18.091228 5105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:44755 every 8 connection(s)
I20260812 11:12:18.095057 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 4910
I20260812 11:12:18.095225 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:12:18.096670 17587 external_mini_cluster.cc:1428] Setting key 5b27f5bc05560f63a07f0a26fdcd7e46
Aug 12 11:12:18 dist-test-slave-drl0 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533137, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:12:18.109083 4679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:18.092916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:55251 (local address 127.17.44.254:45555)
0812 11:12:18.093990 (+ 1074us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:18.093996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:18.094228 (+ 232us) server_negotiation.cc:407] Connection header received
0812 11:12:18.095732 (+ 1504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:18.095736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:18.095794 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:18.095850 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:18.098023 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:18.098894 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:18.100743 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:18.101135 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:18.101240 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:18.104758 (+ 3518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:18.104762 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:18.104765 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:18.104807 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:18.106626 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:18.107293 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:18.107297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:18.107299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:18.107383 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:18.107794 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:18.107798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:18.107800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:18.108020 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:18.108101 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:18.108765 (+ 664us) server_negotiation.cc:299] Negotiation successful
0812 11:12:18.108909 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":972}
I20260812 11:12:18.111119 5108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:18.092194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:45555 (local address 127.17.44.195:55251)
0812 11:12:18.093862 (+ 1668us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:18.093982 (+ 120us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:18.095471 (+ 1489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:18.096277 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:18.096285 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:18.096656 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:18.097818 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:18.097840 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:18.099117 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:18.099122 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:18.100573 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:18.100587 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:18.101470 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:18.101475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:18.101684 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:18.102464 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:18.102489 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:18.104557 (+ 2068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:18.106772 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:18.106777 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:18.106807 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:18.107172 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:18.107492 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:18.107495 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:18.107497 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:18.107670 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:18.108178 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:18.108184 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:18.108488 (+ 304us) client_negotiation.cc:769] Sending connection context
0812 11:12:18.109697 (+ 1209us) client_negotiation.cc:240] Negotiation successful
0812 11:12:18.110054 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1030,"thread_start_us":154,"threads_started":1}
I20260812 11:12:18.112527 5106 heartbeater.cc:347] Connected to a master server at 127.17.44.254:45555
I20260812 11:12:18.112708 5106 heartbeater.cc:464] Registering TS with master...
I20260812 11:12:18.113305 5106 heartbeater.cc:511] Master 127.17.44.254:45555 requested a full tablet report, sending...
I20260812 11:12:18.116277 4351 ts_manager.cc:194] Registered new tserver with Master: 73a22a18776b48a7aefac3121d4949ec (127.17.44.195:44755)
I20260812 11:12:18.117035 4351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.195@KRBTEST.COM'} at 127.17.44.195:55251
I20260812 11:12:18.127218 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:12:18.128026 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 4479
I20260812 11:12:18.170027 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 4684
I20260812 11:12:18.192572 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 4910
I20260812 11:12:18.206733 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 4301
I20260812 11:12:18.217506 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:45555
--webserver_interface=127.17.44.254
--webserver_port=38871
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:45493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:45555
--ranger_config_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.44.212:39715/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.44.212:39715/kms with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:12:18.336119 5110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:18.336377 5110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:18.336426 5110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:18.340587 5110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:12:18.340644 5110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:18.340664 5110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:18.340680 5110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:12:18.340696 5110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:12:18.346630 5110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:45493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.44.212:39715/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:45555
--ranger_config_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:45555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=38871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:18.347716 5110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:18.348737 5110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:18.359558 5116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:18.359589 5118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:18.359558 5122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:18.363071 5110 server_base.cc:1034] running on GCE node
I20260812 11:12:18.363708 5110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:18.365057 5110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:18.366263 5110 hybrid_clock.cc:648] HybridClock initialized: now 1786533138366243 us; error 33 us; skew 500 ppm
Aug 12 11:12:18 dist-test-slave-drl0 krb5kdc[3579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533138, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:18.369619 5110 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:12:18.370642 5110 webserver.cc:460] Webserver started at http://127.17.44.254:38871/ using document root <none> and password file <none>
I20260812 11:12:18.371273 5110 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:18.371346 5110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 11:12:18 dist-test-slave-drl0 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533138, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for HTTP/127.17.44.212@KRBTEST.COM
I20260812 11:12:18.454430 5110 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 11:12:18.457592 5137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:18.459373 5110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 11:12:18.459510 5110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "fcb846f0b6bc4560b181b6c0f574cf13"
format_stamp: "Formatted at 2026-08-12 11:12:16 on dist-test-slave-drl0"
server_key: "963d6e069fb4fa71845884117fb74881"
server_key_iv: "c56a7f403fa032d006f98cfd9b9c3a3e"
server_key_version: "kuduclusterkey@0"
I20260812 11:12:18.460202 5110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:18.509443 5110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:18.513031 5110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:18.513432 5110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:18.521858 5110 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:45555
I20260812 11:12:18.521883 5220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:45555 every 8 connection(s)
I20260812 11:12:18.523609 5110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:12:18.527354 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 5110
I20260812 11:12:18.528052 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.44.212:39715/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:36947
--local_ip_for_outbound_sockets=127.17.44.193
--tserver_master_addrs=127.17.44.254:45555
--webserver_port=39959
--webserver_interface=127.17.44.193
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:45493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.44.212:39715/kms with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:12:18.528717 5221 sys_catalog.cc:263] Verifying existing consensus state
I20260812 11:12:18.530417 5221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13: Bootstrap starting.
I20260812 11:12:18.547514 5221 log.cc:824] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13: Log is configured to *not* fsync() on all Append() calls
I20260812 11:12:18.554281 5221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 11:12:18.554687 5221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13: Bootstrap complete.
I20260812 11:12:18.558838 5221 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb846f0b6bc4560b181b6c0f574cf13" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 45555 } }
I20260812 11:12:18.559332 5221 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb846f0b6bc4560b181b6c0f574cf13, State: Initialized, Role: FOLLOWER
I20260812 11:12:18.559814 5221 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb846f0b6bc4560b181b6c0f574cf13" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 45555 } }
I20260812 11:12:18.559962 5221 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:12:18.560052 5221 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:12:18.560175 5221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [term 1 FOLLOWER]: Advancing to term 2
I20260812 11:12:18.562636 5221 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb846f0b6bc4560b181b6c0f574cf13" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 45555 } }
I20260812 11:12:18.563042 5221 leader_election.cc:302] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcb846f0b6bc4560b181b6c0f574cf13; no voters:
I20260812 11:12:18.563410 5221 leader_election.cc:288] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 11:12:18.563514 5229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 11:12:18.563738 5229 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [term 2 LEADER]: Becoming Leader. State: Replica: fcb846f0b6bc4560b181b6c0f574cf13, State: Running, Role: LEADER
I20260812 11:12:18.564064 5229 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb846f0b6bc4560b181b6c0f574cf13" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 45555 } }
I20260812 11:12:18.564611 5221 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:12:18.565539 5230 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fcb846f0b6bc4560b181b6c0f574cf13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb846f0b6bc4560b181b6c0f574cf13" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 45555 } } }
I20260812 11:12:18.565681 5230 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [sys.catalog]: This master's current role is: LEADER
I20260812 11:12:18.566038 5231 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcb846f0b6bc4560b181b6c0f574cf13. Latest consensus state: current_term: 2 leader_uuid: "fcb846f0b6bc4560b181b6c0f574cf13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb846f0b6bc4560b181b6c0f574cf13" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 45555 } } }
I20260812 11:12:18.566126 5231 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13 [sys.catalog]: This master's current role is: LEADER
I20260812 11:12:18.566423 5240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:12:18.569614 5240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:12:18.570147 5221 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 11:12:18.570278 5240 catalog_manager.cc:1259] Loaded cluster ID: 01d5f806d82c46fd8f935c6cfb7bc3e2
I20260812 11:12:18.570328 5240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:12:18.572628 5240 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:12:18.573267 5240 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13: Loaded TSK: 0
I20260812 11:12:18.574483 5240 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 11:12:18.653855 5224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:18.654182 5224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:18.654247 5224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:18.660246 5224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:18.660356 5224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:18.660499 5224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:12:18.669425 5224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:45493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.44.212:39715/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:36947
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=39959
--tserver_master_addrs=127.17.44.254:45555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:18.670871 5224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:18.672196 5224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:18.686100 5272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:18.686795 5278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:18.686857 5275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:18.688169 5224 server_base.cc:1034] running on GCE node
I20260812 11:12:18.688648 5224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:18.689414 5224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:18.690609 5224 hybrid_clock.cc:648] HybridClock initialized: now 1786533138690587 us; error 38 us; skew 500 ppm
Aug 12 11:12:18 dist-test-slave-drl0 krb5kdc[3579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533138, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:18.694275 5224 init.cc:377] Logged in from keytab as kudu/127.17.44.193@KRBTEST.COM (short username kudu)
I20260812 11:12:18.695415 5224 webserver.cc:460] Webserver started at http://127.17.44.193:39959/ using document root <none> and password file <none>
I20260812 11:12:18.696091 5224 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:18.696202 5224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 11:12:18 dist-test-slave-drl0 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533138, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for HTTP/127.17.44.212@KRBTEST.COM
I20260812 11:12:18.722535 5224 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 11:12:18.726114 5300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:18.727825 5224 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260812 11:12:18.727964 5224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "a3d664ea3f5a4edd9d08173e78cd12d1"
format_stamp: "Formatted at 2026-08-12 11:12:17 on dist-test-slave-drl0"
server_key: "a11da8d76cf34202b2665358b848b8b9"
server_key_iv: "ca14db2487518fdae2e57d254ca308e2"
server_key_version: "kuduclusterkey@0"
I20260812 11:12:18.728698 5224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:18.758792 5224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:18.762674 5224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:18.762867 5224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:18.763588 5224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:12:18.764773 5224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:12:18.764833 5224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:18.764885 5224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:12:18.764914 5224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:18.776548 5224 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:36947
I20260812 11:12:18.778100 5224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:12:18.780375 5475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:36947 every 8 connection(s)
I20260812 11:12:18.781581 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 5224
I20260812 11:12:18.782083 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.44.212:39715/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:36029
--local_ip_for_outbound_sockets=127.17.44.194
--tserver_master_addrs=127.17.44.254:45555
--webserver_port=42519
--webserver_interface=127.17.44.194
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:45493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.44.212:39715/kms with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 11:12:18 dist-test-slave-drl0 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533138, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:12:18.810791 5481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:18.781870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:45555 (local address 127.17.44.193:59709)
0812 11:12:18.782818 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:18.782857 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:18.783982 (+ 1125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:18.785888 (+ 1906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:18.785898 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:18.786252 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:18.787078 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:18.787101 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:18.789107 (+ 2006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:18.789111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:18.790483 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:18.790495 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:18.791767 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:18.791772 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:18.791954 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:18.792700 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:18.792722 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:18.794465 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:18.797159 (+ 2694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:18.797165 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:18.797179 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:18.797537 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:18.797985 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:18.797989 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:18.797991 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:18.798165 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:18.799079 (+ 914us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:18.799085 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:18.799389 (+ 304us) client_negotiation.cc:769] Sending connection context
0812 11:12:18.799602 (+ 213us) client_negotiation.cc:240] Negotiation successful
0812 11:12:18.799932 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":771,"thread_start_us":127,"threads_started":1}
I20260812 11:12:18.812917 5480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:18.781719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:59709 (local address 127.17.44.254:45555)
0812 11:12:18.782551 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:18.782577 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:18.782985 (+ 408us) server_negotiation.cc:407] Connection header received
0812 11:12:18.784564 (+ 1579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:18.784581 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:18.785001 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:18.785348 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:18.787304 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:18.788932 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:18.790632 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:18.791373 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:18.791539 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:18.794718 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:18.794742 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:18.794760 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:18.794792 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:18.797020 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:18.797686 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:18.797692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:18.797698 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:18.797867 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:18.798316 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:18.798320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:18.798321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:18.798834 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:18.799018 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:18.811639 (+ 12621us) server_negotiation.cc:299] Negotiation successful
0812 11:12:18.811990 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":139,"threads_started":1}
I20260812 11:12:18.815496 5477 heartbeater.cc:347] Connected to a master server at 127.17.44.254:45555
I20260812 11:12:18.815749 5477 heartbeater.cc:464] Registering TS with master...
I20260812 11:12:18.816301 5477 heartbeater.cc:511] Master 127.17.44.254:45555 requested a full tablet report, sending...
W20260812 11:12:18.818413 5477 heartbeater.cc:644] Failed to heartbeat to 127.17.44.254:45555 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 11:12:18.818545 5477 heartbeater.cc:464] Registering TS with master...
I20260812 11:12:18.818742 5477 heartbeater.cc:511] Master 127.17.44.254:45555 requested a full tablet report, sending...
I20260812 11:12:18.819633 5477 heartbeater.cc:464] Registering TS with master...
I20260812 11:12:18.819823 5477 heartbeater.cc:511] Master 127.17.44.254:45555 requested a full tablet report, sending...
W20260812 11:12:18.820597 5477 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 11:12:18.952898 5482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:18.953230 5482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:18.953292 5482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:18.959729 5482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:18.959825 5482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:18.960017 5482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:12:18.968647 5482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:45493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.44.212:39715/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:36029
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=42519
--tserver_master_addrs=127.17.44.254:45555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:18.970059 5482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:18.971472 5482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:18.984668 5487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:18.984949 5488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:18.984668 5490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:18.985172 5482 server_base.cc:1034] running on GCE node
I20260812 11:12:18.985740 5482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:18.986567 5482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:18.987780 5482 hybrid_clock.cc:648] HybridClock initialized: now 1786533138987742 us; error 47 us; skew 500 ppm
Aug 12 11:12:18 dist-test-slave-drl0 krb5kdc[3579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533138, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:18.991394 5482 init.cc:377] Logged in from keytab as kudu/127.17.44.194@KRBTEST.COM (short username kudu)
I20260812 11:12:18.992508 5482 webserver.cc:460] Webserver started at http://127.17.44.194:42519/ using document root <none> and password file <none>
I20260812 11:12:18.993255 5482 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:18.993376 5482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 11:12:18 dist-test-slave-drl0 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533138, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for HTTP/127.17.44.212@KRBTEST.COM
I20260812 11:12:19.020895 5482 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 11:12:19.030253 5506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:19.031888 5482 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.002s sys 0.000s
I20260812 11:12:19.032021 5482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "f0fe682fba2f48a8bec6d66bf8ce4e42"
format_stamp: "Formatted at 2026-08-12 11:12:17 on dist-test-slave-drl0"
server_key: "b52de418e946d2b7b8cd457f9da90f86"
server_key_iv: "29b361125d82e8e73a4bb6d027838af7"
server_key_version: "kuduclusterkey@0"
I20260812 11:12:19.032764 5482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:19.070459 5482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:19.074319 5482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:19.074572 5482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:19.075565 5482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:12:19.076870 5482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:12:19.076956 5482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:19.077018 5482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:12:19.077045 5482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:19.089551 5482 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:36029
I20260812 11:12:19.090974 5482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:12:19.095000 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 5482
I20260812 11:12:19.095520 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.44.212:39715/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:44755
--local_ip_for_outbound_sockets=127.17.44.195
--tserver_master_addrs=127.17.44.254:45555
--webserver_port=44683
--webserver_interface=127.17.44.195
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:45493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.44.212:39715/kms with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:12:19.099094 5683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:36029 every 8 connection(s)
Aug 12 11:12:19 dist-test-slave-drl0 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533138, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:12:19.110908 5480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:19.094746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:51707 (local address 127.17.44.254:45555)
0812 11:12:19.094910 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:19.094915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:19.094960 (+ 45us) server_negotiation.cc:407] Connection header received
0812 11:12:19.096360 (+ 1400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:19.096364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:19.096423 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:19.096480 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:19.098248 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:19.099124 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:19.100892 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:19.101266 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:19.101351 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:19.105877 (+ 4526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:19.105881 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:19.105884 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:19.105914 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:19.108043 (+ 2129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:19.108709 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:19.108730 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:19.108732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:19.108830 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:19.109302 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:19.109307 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:19.109309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:19.109552 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:19.109635 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:19.110429 (+ 794us) server_negotiation.cc:299] Negotiation successful
0812 11:12:19.110579 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 11:12:19.112988 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:19.093594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:45555 (local address 127.17.44.194:51707)
0812 11:12:19.094198 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:19.094272 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:19.096078 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:19.096920 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:19.096929 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:19.097314 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:19.098047 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:19.098069 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:19.099309 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:19.099313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:19.100740 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:19.100752 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:19.101575 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:19.101579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:19.101797 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:19.103959 (+ 2162us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:19.103984 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:19.105636 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:19.108189 (+ 2553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:19.108197 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:19.108215 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:19.108574 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:19.108962 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:19.108965 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:19.108967 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:19.109174 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:19.109747 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:19.109754 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:19.110114 (+ 360us) client_negotiation.cc:769] Sending connection context
0812 11:12:19.111376 (+ 1262us) client_negotiation.cc:240] Negotiation successful
0812 11:12:19.111750 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":157,"threads_started":1}
I20260812 11:12:19.114675 5684 heartbeater.cc:347] Connected to a master server at 127.17.44.254:45555
I20260812 11:12:19.114980 5684 heartbeater.cc:464] Registering TS with master...
I20260812 11:12:19.115562 5684 heartbeater.cc:511] Master 127.17.44.254:45555 requested a full tablet report, sending...
W20260812 11:12:19.117853 5684 heartbeater.cc:644] Failed to heartbeat to 127.17.44.254:45555 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 11:12:19.117981 5684 heartbeater.cc:464] Registering TS with master...
I20260812 11:12:19.118139 5684 heartbeater.cc:511] Master 127.17.44.254:45555 requested a full tablet report, sending...
I20260812 11:12:19.119215 5684 heartbeater.cc:464] Registering TS with master...
I20260812 11:12:19.119407 5684 heartbeater.cc:511] Master 127.17.44.254:45555 requested a full tablet report, sending...
W20260812 11:12:19.120276 5684 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 11:12:19.251012 5688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:19.251302 5688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:19.251376 5688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:19.255752 5688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:19.255841 5688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:19.255950 5688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:12:19.262233 5688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:45493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.44.212:39715/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:44755
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=44683
--tserver_master_addrs=127.17.44.254:45555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:19.263536 5688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:19.264644 5688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:19.275770 5699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:19.275777 5695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:19.276321 5697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:19.276757 5688 server_base.cc:1034] running on GCE node
I20260812 11:12:19.277704 5688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:19.278541 5688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:19.280386 5688 hybrid_clock.cc:648] HybridClock initialized: now 1786533139280352 us; error 52 us; skew 500 ppm
Aug 12 11:12:19 dist-test-slave-drl0 krb5kdc[3579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533139, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:19.284662 5688 init.cc:377] Logged in from keytab as kudu/127.17.44.195@KRBTEST.COM (short username kudu)
I20260812 11:12:19.285856 5688 webserver.cc:460] Webserver started at http://127.17.44.195:44683/ using document root <none> and password file <none>
I20260812 11:12:19.286659 5688 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:19.286828 5688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 11:12:19 dist-test-slave-drl0 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533139, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for HTTP/127.17.44.212@KRBTEST.COM
I20260812 11:12:19.318306 5688 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.003s sys 0.000s
I20260812 11:12:19.321873 5709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:19.323597 5688 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 11:12:19.323791 5688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "73a22a18776b48a7aefac3121d4949ec"
format_stamp: "Formatted at 2026-08-12 11:12:17 on dist-test-slave-drl0"
server_key: "710ddf962f7c25498a55200cd7e7546c"
server_key_iv: "2c89e5b7ef527c0e0a1dd7c931caab52"
server_key_version: "kuduclusterkey@0"
I20260812 11:12:19.324831 5688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:19.347667 5688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:19.351305 5688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:19.351766 5688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:19.352537 5688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:12:19.353680 5688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:12:19.353742 5688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:19.353897 5688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:12:19.353940 5688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:19.366541 5688 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:44755
I20260812 11:12:19.366621 5889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:44755 every 8 connection(s)
I20260812 11:12:19.367831 5688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:12:19.370282 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 5688
Aug 12 11:12:19 dist-test-slave-drl0 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533139, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:12:19.387704 5480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:19.371065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38580 (local address 127.17.44.254:45555)
0812 11:12:19.371227 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:19.371233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:19.373689 (+ 2456us) server_negotiation.cc:407] Connection header received
0812 11:12:19.373762 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:19.373766 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:19.373826 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:19.373921 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:19.374974 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:19.375797 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:19.377058 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:19.377399 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:19.377521 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:19.380910 (+ 3389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:19.380918 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:19.380922 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:19.380992 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:19.382747 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:19.384945 (+ 2198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:19.384949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:19.384951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:19.385038 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:19.386261 (+ 1223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:19.386265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:19.386267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:19.386500 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:19.386697 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:19.387369 (+ 672us) server_negotiation.cc:299] Negotiation successful
0812 11:12:19.387529 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61,"spinlock_wait_cycles":35072}
I20260812 11:12:19.393637 5894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:19.371959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:39539 (local address 127.17.44.254:45555)
0812 11:12:19.372223 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:19.372234 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:19.373471 (+ 1237us) server_negotiation.cc:407] Connection header received
0812 11:12:19.376274 (+ 2803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:19.376277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:19.376333 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:19.376432 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:19.377732 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:19.378478 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:19.382891 (+ 4413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:19.383266 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:19.383364 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:19.387282 (+ 3918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:19.387287 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:19.387290 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:19.387344 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:19.390993 (+ 3649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:19.391706 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:19.391710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:19.391712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:19.391812 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:19.392300 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:19.392304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:19.392306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:19.392537 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:19.392651 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:19.393342 (+ 691us) server_negotiation.cc:299] Negotiation successful
0812 11:12:19.393467 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":102,"threads_started":1}
I20260812 11:12:19.394950 5898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:19.372528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:45555 (local address 127.17.44.195:39539)
0812 11:12:19.373275 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:19.373331 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:19.376056 (+ 2725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:19.376612 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:19.376618 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:19.376889 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:19.377565 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:19.377585 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:19.378653 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:19.378657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:19.381862 (+ 3205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:19.381894 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:19.383542 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:19.383546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:19.383747 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:19.384335 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:19.384356 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:19.387035 (+ 2679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:19.391127 (+ 4092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:19.391135 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:19.391154 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:19.391556 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:19.391933 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:19.391937 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:19.391939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:19.392140 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:19.392669 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:19.392677 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:19.393016 (+ 339us) client_negotiation.cc:769] Sending connection context
0812 11:12:19.393247 (+ 231us) client_negotiation.cc:240] Negotiation successful
0812 11:12:19.393602 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":353,"threads_started":1}
I20260812 11:12:19.396304 5890 heartbeater.cc:347] Connected to a master server at 127.17.44.254:45555
I20260812 11:12:19.396560 5890 heartbeater.cc:464] Registering TS with master...
I20260812 11:12:19.397174 5890 heartbeater.cc:511] Master 127.17.44.254:45555 requested a full tablet report, sending...
W20260812 11:12:19.399039 5890 heartbeater.cc:644] Failed to heartbeat to 127.17.44.254:45555 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 11:12:19.399160 5890 heartbeater.cc:464] Registering TS with master...
I20260812 11:12:19.399334 5890 heartbeater.cc:511] Master 127.17.44.254:45555 requested a full tablet report, sending...
I20260812 11:12:19.400600 5890 heartbeater.cc:464] Registering TS with master...
I20260812 11:12:19.400789 5890 heartbeater.cc:511] Master 127.17.44.254:45555 requested a full tablet report, sending...
W20260812 11:12:19.401799 5890 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 11:12:19 dist-test-slave-drl0 krb5kdc[3579](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786533139, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:19.820909 5477 heartbeater.cc:464] Registering TS with master...
I20260812 11:12:19.821250 5477 heartbeater.cc:511] Master 127.17.44.254:45555 requested a full tablet report, sending...
I20260812 11:12:19.822613 5159 ts_manager.cc:194] Registered new tserver with Master: a3d664ea3f5a4edd9d08173e78cd12d1 (127.17.44.193:36947)
I20260812 11:12:19.824411 5159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.193@KRBTEST.COM'} at 127.17.44.193:59709
I20260812 11:12:20.120488 5684 heartbeater.cc:464] Registering TS with master...
I20260812 11:12:20.120810 5684 heartbeater.cc:511] Master 127.17.44.254:45555 requested a full tablet report, sending...
I20260812 11:12:20.121619 5158 ts_manager.cc:194] Registered new tserver with Master: f0fe682fba2f48a8bec6d66bf8ce4e42 (127.17.44.194:36029)
I20260812 11:12:20.122486 5158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.194@KRBTEST.COM'} at 127.17.44.194:51707
Aug 12 11:12:20 dist-test-slave-drl0 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533139, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for HTTP/127.17.44.212@KRBTEST.COM
I20260812 11:12:20.402046 5890 heartbeater.cc:464] Registering TS with master...
I20260812 11:12:20.402388 5890 heartbeater.cc:511] Master 127.17.44.254:45555 requested a full tablet report, sending...
I20260812 11:12:20.403378 5158 ts_manager.cc:194] Registered new tserver with Master: 73a22a18776b48a7aefac3121d4949ec (127.17.44.195:44755)
I20260812 11:12:20.404242 5158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.195@KRBTEST.COM'} at 127.17.44.195:39539
I20260812 11:12:20.411268 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:12:20.428472 5923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:20.412965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38596 (local address 127.17.44.254:45555)
0812 11:12:20.413371 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:20.413376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:20.413394 (+ 18us) server_negotiation.cc:407] Connection header received
0812 11:12:20.413476 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:20.413479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:20.413524 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:20.413622 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:20.414555 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:20.415525 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:20.416807 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:20.417167 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:20.417295 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:20.424124 (+ 6829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:20.424129 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:20.424132 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:20.424163 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:20.426103 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:20.426625 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:20.426629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:20.426631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:20.426732 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:20.427115 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:20.427118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:20.427120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:20.427354 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:20.427483 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:20.428163 (+ 680us) server_negotiation.cc:299] Negotiation successful
0812 11:12:20.428290 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":100,"threads_started":1}
I20260812 11:12:20.433262 5158 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38596:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:12:20.440182 5158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:12:20.465372 5932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:20.455689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:44755 (local address 127.0.0.1:58886)
0812 11:12:20.458622 (+ 2933us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:20.458635 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:20.458709 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:20.459860 (+ 1151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:20.459864 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:20.459886 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:12:20.459985 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:20.459991 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:20.461943 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:20.461949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:20.463596 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:20.463608 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:20.464808 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:20.464812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:20.464948 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:20.464998 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 11:12:20.465093 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 11:12:20.465212 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2864,"thread_start_us":83,"threads_started":1}
I20260812 11:12:20.466341 5941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:20.456087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58886 (local address 127.17.44.195:44755)
0812 11:12:20.457857 (+ 1770us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:20.457865 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:20.459202 (+ 1337us) server_negotiation.cc:407] Connection header received
0812 11:12:20.459292 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:20.459297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:20.459480 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:20.460210 (+ 730us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:12:20.460399 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:20.461747 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:20.463764 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:20.464644 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:20.465887 (+ 1243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:20.466047 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:20.466102 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 11:12:20.466198 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1565,"thread_start_us":121,"threads_started":1}
I20260812 11:12:20.469055 5935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:20.453497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33372 (local address 127.17.44.194:36029)
0812 11:12:20.458989 (+ 5492us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:20.458996 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:20.459018 (+ 22us) server_negotiation.cc:407] Connection header received
0812 11:12:20.459089 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:20.459094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:20.459253 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:20.459407 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:12:20.459955 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:20.461121 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:20.467421 (+ 6300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:20.468220 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:20.468316 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:20.468447 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:20.468807 (+ 360us) server_negotiation.cc:299] Negotiation successful
0812 11:12:20.468911 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5353,"thread_start_us":92,"threads_started":1}
I20260812 11:12:20.469590 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:20.453367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:36029 (local address 127.0.0.1:33372)
0812 11:12:20.457155 (+ 3788us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:20.457209 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:20.457344 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:20.459644 (+ 2300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:20.459650 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:20.459674 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:12:20.459820 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:20.459827 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:20.461371 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:20.461375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:20.462991 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:20.463004 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:20.468552 (+ 5548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:20.468555 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:20.468651 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:20.468674 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 11:12:20.469385 (+ 711us) client_negotiation.cc:240] Negotiation successful
0812 11:12:20.469465 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3600,"thread_start_us":98,"threads_started":1}
I20260812 11:12:20.470424 5936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:20.454645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:36947 (local address 127.0.0.1:59946)
0812 11:12:20.458150 (+ 3505us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:20.458168 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:20.458406 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:20.463310 (+ 4904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:20.463315 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:20.463337 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:12:20.463428 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:20.463434 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:20.465015 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:20.465019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:20.466261 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:20.466274 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:20.470057 (+ 3783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:20.470061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:20.470160 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:20.470183 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 11:12:20.470232 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 11:12:20.470304 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3419,"thread_start_us":94,"threads_started":1}
I20260812 11:12:20.470507 5789 tablet_service.cc:1467] Processing CreateTablet for tablet 21a3c25eba4d489aab60edfa25c50544 (DEFAULT_TABLE table=test-table [id=7bff748b1085470dbc83cc4b4564a06a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:12:20.471366 5938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:20.454972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59946 (local address 127.17.44.193:36947)
0812 11:12:20.457948 (+ 2976us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:20.457957 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:20.458427 (+ 470us) server_negotiation.cc:407] Connection header received
0812 11:12:20.458687 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:20.458694 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:20.458984 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:20.463576 (+ 4592us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:12:20.463708 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:20.464885 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:20.466476 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:20.467269 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:20.470847 (+ 3578us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:20.471008 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:20.471058 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 11:12:20.471217 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2797,"thread_start_us":72,"threads_started":1}
I20260812 11:12:20.471803 5789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21a3c25eba4d489aab60edfa25c50544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:12:20.472465 5558 tablet_service.cc:1467] Processing CreateTablet for tablet 21a3c25eba4d489aab60edfa25c50544 (DEFAULT_TABLE table=test-table [id=7bff748b1085470dbc83cc4b4564a06a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:12:20.473383 5558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21a3c25eba4d489aab60edfa25c50544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:12:20.473938 5380 tablet_service.cc:1467] Processing CreateTablet for tablet 21a3c25eba4d489aab60edfa25c50544 (DEFAULT_TABLE table=test-table [id=7bff748b1085470dbc83cc4b4564a06a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:12:20.474815 5380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21a3c25eba4d489aab60edfa25c50544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:12:20.487016 5948 tablet_bootstrap.cc:492] T 21a3c25eba4d489aab60edfa25c50544 P 73a22a18776b48a7aefac3121d4949ec: Bootstrap starting.
I20260812 11:12:20.487089 5945 tablet_bootstrap.cc:492] T 21a3c25eba4d489aab60edfa25c50544 P f0fe682fba2f48a8bec6d66bf8ce4e42: Bootstrap starting.
Aug 12 11:12:20 dist-test-slave-drl0 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533139, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for HTTP/127.17.44.212@KRBTEST.COM
I20260812 11:12:20.489485 5945 tablet_bootstrap.cc:654] T 21a3c25eba4d489aab60edfa25c50544 P f0fe682fba2f48a8bec6d66bf8ce4e42: Neither blocks nor log segments found. Creating new log.
I20260812 11:12:20.489600 5948 tablet_bootstrap.cc:654] T 21a3c25eba4d489aab60edfa25c50544 P 73a22a18776b48a7aefac3121d4949ec: Neither blocks nor log segments found. Creating new log.
I20260812 11:12:20.490470 5948 log.cc:824] T 21a3c25eba4d489aab60edfa25c50544 P 73a22a18776b48a7aefac3121d4949ec: Log is configured to *not* fsync() on all Append() calls
I20260812 11:12:20.490482 5945 log.cc:824] T 21a3c25eba4d489aab60edfa25c50544 P f0fe682fba2f48a8bec6d66bf8ce4e42: Log is configured to *not* fsync() on all Append() calls
I20260812 11:12:20.491232 5946 tablet_bootstrap.cc:492] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1: Bootstrap starting.
I20260812 11:12:20.492874 5948 tablet_bootstrap.cc:492] T 21a3c25eba4d489aab60edfa25c50544 P 73a22a18776b48a7aefac3121d4949ec: No bootstrap required, opened a new log
I20260812 11:12:20.493079 5948 ts_tablet_manager.cc:1397] T 21a3c25eba4d489aab60edfa25c50544 P 73a22a18776b48a7aefac3121d4949ec: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:12:20.494515 5946 tablet_bootstrap.cc:654] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1: Neither blocks nor log segments found. Creating new log.
I20260812 11:12:20.496095 5946 log.cc:824] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1: Log is configured to *not* fsync() on all Append() calls
I20260812 11:12:20.498898 5946 tablet_bootstrap.cc:492] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1: No bootstrap required, opened a new log
I20260812 11:12:20.499190 5946 ts_tablet_manager.cc:1397] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260812 11:12:20.499454 5945 tablet_bootstrap.cc:492] T 21a3c25eba4d489aab60edfa25c50544 P f0fe682fba2f48a8bec6d66bf8ce4e42: No bootstrap required, opened a new log
I20260812 11:12:20.499683 5945 ts_tablet_manager.cc:1397] T 21a3c25eba4d489aab60edfa25c50544 P f0fe682fba2f48a8bec6d66bf8ce4e42: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.002s
I20260812 11:12:20.503975 5946 raft_consensus.cc:348] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0fe682fba2f48a8bec6d66bf8ce4e42" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 36029 } } peers { permanent_uuid: "a3d664ea3f5a4edd9d08173e78cd12d1" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36947 } } peers { permanent_uuid: "73a22a18776b48a7aefac3121d4949ec" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 44755 } }
I20260812 11:12:20.504412 5946 raft_consensus.cc:374] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:12:20.504554 5946 raft_consensus.cc:729] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d664ea3f5a4edd9d08173e78cd12d1, State: Initialized, Role: FOLLOWER
I20260812 11:12:20.504910 5945 raft_consensus.cc:348] T 21a3c25eba4d489aab60edfa25c50544 P f0fe682fba2f48a8bec6d66bf8ce4e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0fe682fba2f48a8bec6d66bf8ce4e42" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 36029 } } peers { permanent_uuid: "a3d664ea3f5a4edd9d08173e78cd12d1" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36947 } } peers { permanent_uuid: "73a22a18776b48a7aefac3121d4949ec" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 44755 } }
I20260812 11:12:20.505152 5945 raft_consensus.cc:374] T 21a3c25eba4d489aab60edfa25c50544 P f0fe682fba2f48a8bec6d66bf8ce4e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:12:20.505206 5945 raft_consensus.cc:729] T 21a3c25eba4d489aab60edfa25c50544 P f0fe682fba2f48a8bec6d66bf8ce4e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0fe682fba2f48a8bec6d66bf8ce4e42, State: Initialized, Role: FOLLOWER
I20260812 11:12:20.505308 5946 consensus_queue.cc:260] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0fe682fba2f48a8bec6d66bf8ce4e42" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 36029 } } peers { permanent_uuid: "a3d664ea3f5a4edd9d08173e78cd12d1" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36947 } } peers { permanent_uuid: "73a22a18776b48a7aefac3121d4949ec" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 44755 } }
I20260812 11:12:20.505646 5945 consensus_queue.cc:260] T 21a3c25eba4d489aab60edfa25c50544 P f0fe682fba2f48a8bec6d66bf8ce4e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0fe682fba2f48a8bec6d66bf8ce4e42" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 36029 } } peers { permanent_uuid: "a3d664ea3f5a4edd9d08173e78cd12d1" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36947 } } peers { permanent_uuid: "73a22a18776b48a7aefac3121d4949ec" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 44755 } }
I20260812 11:12:20.506556 5946 ts_tablet_manager.cc:1428] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260812 11:12:20.506603 5945 ts_tablet_manager.cc:1428] T 21a3c25eba4d489aab60edfa25c50544 P f0fe682fba2f48a8bec6d66bf8ce4e42: Time spent starting tablet: real 0.007s user 0.003s sys 0.001s
I20260812 11:12:20.507019 5684 heartbeater.cc:503] Master 127.17.44.254:45555 was elected leader, sending a full tablet report...
I20260812 11:12:20.508119 5477 heartbeater.cc:503] Master 127.17.44.254:45555 was elected leader, sending a full tablet report...
I20260812 11:12:20.513226 5948 raft_consensus.cc:348] T 21a3c25eba4d489aab60edfa25c50544 P 73a22a18776b48a7aefac3121d4949ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0fe682fba2f48a8bec6d66bf8ce4e42" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 36029 } } peers { permanent_uuid: "a3d664ea3f5a4edd9d08173e78cd12d1" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36947 } } peers { permanent_uuid: "73a22a18776b48a7aefac3121d4949ec" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 44755 } }
I20260812 11:12:20.513545 5948 raft_consensus.cc:374] T 21a3c25eba4d489aab60edfa25c50544 P 73a22a18776b48a7aefac3121d4949ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:12:20.513593 5948 raft_consensus.cc:729] T 21a3c25eba4d489aab60edfa25c50544 P 73a22a18776b48a7aefac3121d4949ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73a22a18776b48a7aefac3121d4949ec, State: Initialized, Role: FOLLOWER
I20260812 11:12:20.514021 5948 consensus_queue.cc:260] T 21a3c25eba4d489aab60edfa25c50544 P 73a22a18776b48a7aefac3121d4949ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0fe682fba2f48a8bec6d66bf8ce4e42" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 36029 } } peers { permanent_uuid: "a3d664ea3f5a4edd9d08173e78cd12d1" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36947 } } peers { permanent_uuid: "73a22a18776b48a7aefac3121d4949ec" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 44755 } }
I20260812 11:12:20.514730 5890 heartbeater.cc:503] Master 127.17.44.254:45555 was elected leader, sending a full tablet report...
I20260812 11:12:20.515235 5948 ts_tablet_manager.cc:1428] T 21a3c25eba4d489aab60edfa25c50544 P 73a22a18776b48a7aefac3121d4949ec: Time spent starting tablet: real 0.022s user 0.006s sys 0.000s
W20260812 11:12:20.530776 5479 tablet.cc:2369] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:12:20.549713 5954 raft_consensus.cc:482] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:12:20.549943 5954 raft_consensus.cc:504] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0fe682fba2f48a8bec6d66bf8ce4e42" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 36029 } } peers { permanent_uuid: "a3d664ea3f5a4edd9d08173e78cd12d1" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36947 } } peers { permanent_uuid: "73a22a18776b48a7aefac3121d4949ec" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 44755 } }
I20260812 11:12:20.551438 5954 leader_election.cc:288] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0fe682fba2f48a8bec6d66bf8ce4e42 (127.17.44.194:36029), 73a22a18776b48a7aefac3121d4949ec (127.17.44.195:44755)
I20260812 11:12:20.558593 5941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:20.552513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:42453 (local address 127.17.44.195:44755)
0812 11:12:20.552664 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:20.552670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:20.553968 (+ 1298us) server_negotiation.cc:407] Connection header received
0812 11:12:20.554162 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:20.554166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:20.554214 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:20.554263 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:12:20.554673 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:20.555501 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:20.557257 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:20.557841 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:20.557954 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:20.558009 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:20.558372 (+ 363us) server_negotiation.cc:299] Negotiation successful
0812 11:12:20.558454 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 11:12:20.558801 5935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:20.552584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:56625 (local address 127.17.44.194:36029)
0812 11:12:20.552783 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:20.552787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:20.553594 (+ 807us) server_negotiation.cc:407] Connection header received
0812 11:12:20.553741 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:20.553745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:20.553799 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:20.553881 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:12:20.554388 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:20.555192 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:20.557444 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:20.558022 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:20.558125 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:20.558194 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:20.558574 (+ 380us) server_negotiation.cc:299] Negotiation successful
0812 11:12:20.558669 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260812 11:12:20.559028 5958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:20.552071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:44755 (local address 127.17.44.193:42453)
0812 11:12:20.553876 (+ 1805us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:20.553888 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:20.554041 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:20.554400 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:20.554403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:20.554419 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:12:20.554543 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:20.554549 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:20.555652 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:20.555655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:20.557111 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:20.557123 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:20.558117 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:20.558120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:20.558223 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:20.558246 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 11:12:20.558747 (+ 501us) client_negotiation.cc:240] Negotiation successful
0812 11:12:20.558825 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1743,"thread_start_us":84,"threads_started":1}
I20260812 11:12:20.560091 5818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a3c25eba4d489aab60edfa25c50544" candidate_uuid: "a3d664ea3f5a4edd9d08173e78cd12d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73a22a18776b48a7aefac3121d4949ec" is_pre_election: true
I20260812 11:12:20.560448 5818 raft_consensus.cc:2454] T 21a3c25eba4d489aab60edfa25c50544 P 73a22a18776b48a7aefac3121d4949ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d664ea3f5a4edd9d08173e78cd12d1 in term 0.
I20260812 11:12:20.561060 5305 leader_election.cc:302] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73a22a18776b48a7aefac3121d4949ec, a3d664ea3f5a4edd9d08173e78cd12d1; no voters:
I20260812 11:12:20.561395 5954 raft_consensus.cc:2781] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:12:20.561478 5954 raft_consensus.cc:482] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:12:20.561534 5954 raft_consensus.cc:3037] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:12:20.563459 5954 raft_consensus.cc:504] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0fe682fba2f48a8bec6d66bf8ce4e42" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 36029 } } peers { permanent_uuid: "a3d664ea3f5a4edd9d08173e78cd12d1" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36947 } } peers { permanent_uuid: "73a22a18776b48a7aefac3121d4949ec" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 44755 } }
I20260812 11:12:20.563845 5954 leader_election.cc:288] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1 [CANDIDATE]: Term 1 election: Requested vote from peers f0fe682fba2f48a8bec6d66bf8ce4e42 (127.17.44.194:36029), 73a22a18776b48a7aefac3121d4949ec (127.17.44.195:44755)
I20260812 11:12:20.564361 5818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a3c25eba4d489aab60edfa25c50544" candidate_uuid: "a3d664ea3f5a4edd9d08173e78cd12d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73a22a18776b48a7aefac3121d4949ec"
I20260812 11:12:20.564421 5959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:20.551669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:36029 (local address 127.17.44.193:56625)
0812 11:12:20.553499 (+ 1830us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:20.553515 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:20.553640 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:20.554010 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:20.554013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:20.554079 (+ 66us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:12:20.554238 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:20.554244 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:20.555355 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:20.555358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:20.556932 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:20.556943 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:20.558308 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:20.558311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:20.558415 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:20.558440 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 11:12:20.564180 (+ 5740us) client_negotiation.cc:240] Negotiation successful
0812 11:12:20.564260 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1748,"spinlock_wait_cycles":19328,"thread_start_us":140,"threads_started":1}
I20260812 11:12:20.564492 5818 raft_consensus.cc:3037] T 21a3c25eba4d489aab60edfa25c50544 P 73a22a18776b48a7aefac3121d4949ec [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:12:20.565928 5818 raft_consensus.cc:2454] T 21a3c25eba4d489aab60edfa25c50544 P 73a22a18776b48a7aefac3121d4949ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d664ea3f5a4edd9d08173e78cd12d1 in term 1.
I20260812 11:12:20.566075 5586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a3c25eba4d489aab60edfa25c50544" candidate_uuid: "a3d664ea3f5a4edd9d08173e78cd12d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0fe682fba2f48a8bec6d66bf8ce4e42" is_pre_election: true
I20260812 11:12:20.566316 5305 leader_election.cc:302] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73a22a18776b48a7aefac3121d4949ec, a3d664ea3f5a4edd9d08173e78cd12d1; no voters:
I20260812 11:12:20.566392 5586 raft_consensus.cc:2454] T 21a3c25eba4d489aab60edfa25c50544 P f0fe682fba2f48a8bec6d66bf8ce4e42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3d664ea3f5a4edd9d08173e78cd12d1 in term 0.
I20260812 11:12:20.566481 5954 raft_consensus.cc:2781] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:12:20.566682 5954 raft_consensus.cc:686] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1 [term 1 LEADER]: Becoming Leader. State: Replica: a3d664ea3f5a4edd9d08173e78cd12d1, State: Running, Role: LEADER
I20260812 11:12:20.566751 5589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21a3c25eba4d489aab60edfa25c50544" candidate_uuid: "a3d664ea3f5a4edd9d08173e78cd12d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0fe682fba2f48a8bec6d66bf8ce4e42"
I20260812 11:12:20.566847 5589 raft_consensus.cc:3037] T 21a3c25eba4d489aab60edfa25c50544 P f0fe682fba2f48a8bec6d66bf8ce4e42 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:12:20.567123 5954 consensus_queue.cc:237] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0fe682fba2f48a8bec6d66bf8ce4e42" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 36029 } } peers { permanent_uuid: "a3d664ea3f5a4edd9d08173e78cd12d1" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36947 } } peers { permanent_uuid: "73a22a18776b48a7aefac3121d4949ec" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 44755 } }
I20260812 11:12:20.568111 5589 raft_consensus.cc:2454] T 21a3c25eba4d489aab60edfa25c50544 P f0fe682fba2f48a8bec6d66bf8ce4e42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3d664ea3f5a4edd9d08173e78cd12d1 in term 1.
I20260812 11:12:20.570747 5159 catalog_manager.cc:5495] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3d664ea3f5a4edd9d08173e78cd12d1 (127.17.44.193). New cstate: current_term: 1 leader_uuid: "a3d664ea3f5a4edd9d08173e78cd12d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0fe682fba2f48a8bec6d66bf8ce4e42" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 36029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3d664ea3f5a4edd9d08173e78cd12d1" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 36947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73a22a18776b48a7aefac3121d4949ec" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 44755 } health_report { overall_health: UNKNOWN } } }
I20260812 11:12:20.600422 5938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:20.595436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59958 (local address 127.17.44.193:36947)
0812 11:12:20.595752 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:20.595760 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:20.595790 (+ 30us) server_negotiation.cc:407] Connection header received
0812 11:12:20.595939 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:20.595944 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:20.596051 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:20.596274 (+ 223us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:12:20.596874 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:20.597630 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:20.598537 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:20.598769 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:20.598893 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:20.599256 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:12:20.599893 (+ 637us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:12:20.600029 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:20.600083 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 11:12:20.600146 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260812 11:12:20.601040 5685 tablet.cc:2369] T 21a3c25eba4d489aab60edfa25c50544 P f0fe682fba2f48a8bec6d66bf8ce4e42: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:12:20.608996 5818 raft_consensus.cc:1261] T 21a3c25eba4d489aab60edfa25c50544 P 73a22a18776b48a7aefac3121d4949ec [term 1 FOLLOWER]: Refusing update from remote peer a3d664ea3f5a4edd9d08173e78cd12d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:12:20.609833 5954 consensus_queue.cc:1035] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73a22a18776b48a7aefac3121d4949ec" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:12:20.620564 5589 raft_consensus.cc:1261] T 21a3c25eba4d489aab60edfa25c50544 P f0fe682fba2f48a8bec6d66bf8ce4e42 [term 1 FOLLOWER]: Refusing update from remote peer a3d664ea3f5a4edd9d08173e78cd12d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260812 11:12:20.622496 5891 tablet.cc:2369] T 21a3c25eba4d489aab60edfa25c50544 P 73a22a18776b48a7aefac3121d4949ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:12:20.621320 5969 mvcc.cc:205] Tried to move back new op lower bound from 7317639743923929088 to 7317639743766216704. Current Snapshot: MvccSnapshot[applied={T|T < 7317639743923929088}]
I20260812 11:12:20.621662 5954 consensus_queue.cc:1035] T 21a3c25eba4d489aab60edfa25c50544 P a3d664ea3f5a4edd9d08173e78cd12d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0fe682fba2f48a8bec6d66bf8ce4e42" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 36029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260812 11:12:20.625973 5970 mvcc.cc:205] Tried to move back new op lower bound from 7317639743923929088 to 7317639743766216704. Current Snapshot: MvccSnapshot[applied={T|T < 7317639743923929088}]
I20260812 11:12:20.638821 5159 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:12:20.639055 5159 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:12:20.642514 5159 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fcb846f0b6bc4560b181b6c0f574cf13: Sending DeleteTablet for 3 replicas of tablet 21a3c25eba4d489aab60edfa25c50544
I20260812 11:12:20.643733 5380 tablet_service.cc:1514] Processing DeleteTablet for tablet 21a3c25eba4d489aab60edfa25c50544 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:12:20 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:59946
I20260812 11:12:20.644130 5558 tablet_service.cc:1514] Processing DeleteTablet for tablet 21a3c25eba4d489aab60edfa25c50544 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:12:20 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:33372
I20260812 11:12:20.644466 5789 tablet_service.cc:1514] Processing DeleteTablet for tablet 21a3c25eba4d489aab60edfa25c50544 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:12:20 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:58886
I20260812 11:12:20.645303 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 5224
I20260812 11:12:20.655184 5984 tablet_replica.cc:330] T 21a3c25eba4d489aab60edfa25c50544 P f0fe682fba2f48a8bec6d66bf8ce4e42: stopping tablet replica
I20260812 11:12:20.655615 5984 raft_consensus.cc:2229] T 21a3c25eba4d489aab60edfa25c50544 P f0fe682fba2f48a8bec6d66bf8ce4e42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:12:20.656008 5984 raft_consensus.cc:2258] T 21a3c25eba4d489aab60edfa25c50544 P f0fe682fba2f48a8bec6d66bf8ce4e42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:12:20.656858 5985 tablet_replica.cc:330] T 21a3c25eba4d489aab60edfa25c50544 P 73a22a18776b48a7aefac3121d4949ec: stopping tablet replica
I20260812 11:12:20.657934 5984 ts_tablet_manager.cc:1905] T 21a3c25eba4d489aab60edfa25c50544 P f0fe682fba2f48a8bec6d66bf8ce4e42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:12:20.658542 5985 raft_consensus.cc:2229] T 21a3c25eba4d489aab60edfa25c50544 P 73a22a18776b48a7aefac3121d4949ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:12:20.659068 5985 raft_consensus.cc:2258] T 21a3c25eba4d489aab60edfa25c50544 P 73a22a18776b48a7aefac3121d4949ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:12:20.662057 5984 ts_tablet_manager.cc:1918] T 21a3c25eba4d489aab60edfa25c50544 P f0fe682fba2f48a8bec6d66bf8ce4e42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:12:20.662187 5984 log.cc:1196] T 21a3c25eba4d489aab60edfa25c50544 P f0fe682fba2f48a8bec6d66bf8ce4e42: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/21a3c25eba4d489aab60edfa25c50544
I20260812 11:12:20.662595 5984 ts_tablet_manager.cc:1939] T 21a3c25eba4d489aab60edfa25c50544 P f0fe682fba2f48a8bec6d66bf8ce4e42: Deleting consensus metadata
I20260812 11:12:20.663921 5985 ts_tablet_manager.cc:1905] T 21a3c25eba4d489aab60edfa25c50544 P 73a22a18776b48a7aefac3121d4949ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:12:20.664098 5140 catalog_manager.cc:4841] TS f0fe682fba2f48a8bec6d66bf8ce4e42 (127.17.44.194:36029): tablet 21a3c25eba4d489aab60edfa25c50544 (table test-table [id=7bff748b1085470dbc83cc4b4564a06a]) successfully deleted
I20260812 11:12:20.670768 5985 ts_tablet_manager.cc:1918] T 21a3c25eba4d489aab60edfa25c50544 P 73a22a18776b48a7aefac3121d4949ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:12:20.670989 5985 log.cc:1196] T 21a3c25eba4d489aab60edfa25c50544 P 73a22a18776b48a7aefac3121d4949ec: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/21a3c25eba4d489aab60edfa25c50544
I20260812 11:12:20.671455 5985 ts_tablet_manager.cc:1939] T 21a3c25eba4d489aab60edfa25c50544 P 73a22a18776b48a7aefac3121d4949ec: Deleting consensus metadata
I20260812 11:12:20.672978 5140 catalog_manager.cc:4841] TS 73a22a18776b48a7aefac3121d4949ec (127.17.44.195:44755): tablet 21a3c25eba4d489aab60edfa25c50544 (table test-table [id=7bff748b1085470dbc83cc4b4564a06a]) successfully deleted
W20260812 11:12:20.675036 5139 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.44.193:36947 (error 108)
I20260812 11:12:20.676615 5936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:20.676303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:36947 (local address 127.0.0.1:59966)
0812 11:12:20.676433 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:20.676520 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.193:36947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 11:12:20.677078 5139 catalog_manager.cc:4584] TS a3d664ea3f5a4edd9d08173e78cd12d1 (127.17.44.193:36947): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 21a3c25eba4d489aab60edfa25c50544: Network error: Client connection negotiation failed: client connection to 127.17.44.193:36947: connect: Connection refused (error 111)
I20260812 11:12:20.677692 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 5482
I20260812 11:12:20.722658 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 5688
I20260812 11:12:20.731269 5936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:20.730817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:36947 (local address 127.0.0.1:59968)
0812 11:12:20.731072 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:20.731144 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.193:36947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260812 11:12:20.732154 5936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:20.731833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:36947 (local address 127.0.0.1:59974)
0812 11:12:20.732005 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:20.732067 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.193:36947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260812 11:12:20.733289 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 5110
I20260812 11:12:20.747728 17587 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 11:12:21.093225 17587 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 11:12:21.093345 17587 mini_ranger.cc:66] Stopping Ranger...
I20260812 11:12:21.449230 17587 mini_ranger.cc:68] Stopped Ranger
2026-08-12 11:12:21.449 UTC [3622] LOG: received smart shutdown request
2026-08-12 11:12:21.454 UTC [3622] LOG: background worker "logical replication launcher" (PID 3635) exited with exit code 1
2026-08-12 11:12:21.454 UTC [3628] LOG: shutting down
2026-08-12 11:12:21.659 UTC [3622] LOG: database system is shut down
2026-08-12T11:12:21Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53742 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:12:21 dist-test-slave-drl0 krb5kdc[6002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:12:21 dist-test-slave-drl0 krb5kdc[6002](info): set up 2 sockets
Aug 12 11:12:21 dist-test-slave-drl0 krb5kdc[6002](info): commencing operation
krb5kdc: starting...
W20260812 11:12:23.105012 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.287s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:12:23 dist-test-slave-drl0 krb5kdc[6002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:12:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:12:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:12:23.238503 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:43701
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:39933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:43701,127.17.44.253:35715,127.17.44.252:42583
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:12:23.359985 6019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:23.360231 6019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:23.360270 6019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:23.364317 6019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:12:23.364375 6019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:23.364394 6019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:23.364418 6019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:12:23.364434 6019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:12:23.374552 6019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:39933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:43701,127.17.44.253:35715,127.17.44.252:42583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:43701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:23.376336 6019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:23.377825 6019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:23.387027 6027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:23.387087 6025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:23.387035 6024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:23.387123 6019 server_base.cc:1034] running on GCE node
I20260812 11:12:23.388144 6019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:23.389577 6019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:23.390894 6019 hybrid_clock.cc:648] HybridClock initialized: now 1786533143390862 us; error 43 us; skew 500 ppm
Aug 12 11:12:23 dist-test-slave-drl0 krb5kdc[6002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533143, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:23.394683 6019 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:12:23.395687 6019 webserver.cc:460] Webserver started at http://127.17.44.254:34303/ using document root <none> and password file <none>
I20260812 11:12:23.396265 6019 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:23.396353 6019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:12:23.396593 6019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:12:23.398612 6019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "e1b0e89a66294cdb82ade6b8efc2a92a"
format_stamp: "Formatted at 2026-08-12 11:12:23 on dist-test-slave-drl0"
server_key: "793fe9d0538a179f8203f82132d6fb4f"
server_key_iv: "bbc6c7601b68baec954493770ffc9e8d"
server_key_version: "encryptionkey@0"
I20260812 11:12:23.399204 6019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "e1b0e89a66294cdb82ade6b8efc2a92a"
format_stamp: "Formatted at 2026-08-12 11:12:23 on dist-test-slave-drl0"
server_key: "793fe9d0538a179f8203f82132d6fb4f"
server_key_iv: "bbc6c7601b68baec954493770ffc9e8d"
server_key_version: "encryptionkey@0"
I20260812 11:12:23.402825 6019 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260812 11:12:23.405441 6036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:23.406730 6019 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 11:12:23.406886 6019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "e1b0e89a66294cdb82ade6b8efc2a92a"
format_stamp: "Formatted at 2026-08-12 11:12:23 on dist-test-slave-drl0"
server_key: "793fe9d0538a179f8203f82132d6fb4f"
server_key_iv: "bbc6c7601b68baec954493770ffc9e8d"
server_key_version: "encryptionkey@0"
I20260812 11:12:23.407081 6019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:23.417461 6019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:23.420712 6019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:23.420913 6019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:23.428416 6019 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:43701
I20260812 11:12:23.428440 6118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:43701 every 8 connection(s)
I20260812 11:12:23.429333 6019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:12:23.432188 6121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:12:23.435974 6121 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43701 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:12:23.436695 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 6019
I20260812 11:12:23.436831 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:12:23.437216 17587 external_mini_cluster.cc:1428] Setting key 5315c3fa79a03db5a829d20b18fcd165
Aug 12 11:12:23 dist-test-slave-drl0 krb5kdc[6002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533143, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.44.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.253@KRBTEST.COM" created.
I20260812 11:12:23.453855 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.438582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36508 (local address 127.17.44.254:43701)
0812 11:12:23.439149 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:23.439163 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:23.439211 (+ 48us) server_negotiation.cc:407] Connection header received
0812 11:12:23.440219 (+ 1008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:23.440223 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:23.440385 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:23.440567 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:23.441572 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.442581 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.444242 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.444767 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.444897 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:23.447881 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:23.447886 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:23.447896 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:23.447917 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:23.450014 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.450549 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.450560 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.450561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.450660 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.451031 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.451034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.451036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.451488 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:23.451641 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:23.452569 (+ 928us) server_negotiation.cc:299] Negotiation successful
0812 11:12:23.452979 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":78,"server-negotiator.queue_time_us":441,"spinlock_wait_cycles":17152,"thread_start_us":157,"threads_started":1}
I20260812 11:12:23.453855 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.438582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:43701 (local address 127.0.0.1:36508)
0812 11:12:23.439061 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.439113 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:23.439932 (+ 819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:23.440566 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:23.440572 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:23.441012 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:23.441378 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:23.441410 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.442753 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.442760 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:23.444095 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:23.444110 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.444956 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.444962 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:23.445111 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:23.445728 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:23.445753 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:23.447715 (+ 1962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:23.450177 (+ 2462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:23.450186 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:23.450195 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:23.450426 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:23.450770 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:23.450775 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:23.450777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:23.450889 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:23.451811 (+ 922us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:23.451824 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:23.452196 (+ 372us) client_negotiation.cc:769] Sending connection context
0812 11:12:23.452405 (+ 209us) client_negotiation.cc:240] Negotiation successful
0812 11:12:23.452807 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":139,"threads_started":1}
I20260812 11:12:23.456929 6121 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:12:23.457780 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.457429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:35715 (local address 127.0.0.1:53732)
0812 11:12:23.457589 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.457688 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.253:35715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 11:12:23.458184 6039 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.44.253:35715: connect: Connection refused (error 111)
I20260812 11:12:23.459358 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.459099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:35715 (local address 127.0.0.1:53734)
0812 11:12:23.459226 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.459286 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.253:35715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 11:12:23.459609 6121 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.253:35715: Network error: Client connection negotiation failed: client connection to 127.17.44.253:35715: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.253.keytab.
Entry for principal kudu/127.17.44.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.253.keytab.
I20260812 11:12:23.471809 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.253:35715
--webserver_interface=127.17.44.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.253.keytab
--principal=kudu/127.17.44.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:39933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:43701,127.17.44.253:35715,127.17.44.252:42583
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:12:23.508968 6121 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } attempt: 1
I20260812 11:12:23.509986 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.509654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:35715 (local address 127.0.0.1:53736)
0812 11:12:23.509837 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.509903 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.253:35715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260812 11:12:23.510740 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.510477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:35715 (local address 127.0.0.1:53746)
0812 11:12:23.510586 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.510670 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.253:35715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 11:12:23.510892 6121 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.253:35715: Network error: Client connection negotiation failed: client connection to 127.17.44.253:35715: connect: Connection refused (error 111)
I20260812 11:12:23.579092 6121 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } attempt: 2
I20260812 11:12:23.579941 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.579656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:35715 (local address 127.0.0.1:53748)
0812 11:12:23.579796 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.579862 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.253:35715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 11:12:23.580745 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.580503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:35715 (local address 127.0.0.1:53754)
0812 11:12:23.580636 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.580687 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.253:35715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260812 11:12:23.581027 6121 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.253:35715: Network error: Client connection negotiation failed: client connection to 127.17.44.253:35715: connect: Connection refused (error 111)
W20260812 11:12:23.592724 6131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:23.592965 6131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:23.593046 6131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:23.597436 6131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:12:23.597487 6131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:23.597507 6131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:23.597523 6131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:12:23.597577 6131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:12:23.602758 6131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:39933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:43701,127.17.44.253:35715,127.17.44.252:42583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.253.keytab
--principal=kudu/127.17.44.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.253:35715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.44.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:23.604192 6131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:23.605520 6131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:23.615170 6136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:23.615183 6142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:23.615250 6140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:23.615232 6131 server_base.cc:1034] running on GCE node
I20260812 11:12:23.616053 6131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:23.617228 6131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:23.618454 6131 hybrid_clock.cc:648] HybridClock initialized: now 1786533143618426 us; error 40 us; skew 500 ppm
Aug 12 11:12:23 dist-test-slave-drl0 krb5kdc[6002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533143, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:23.621740 6131 init.cc:377] Logged in from keytab as kudu/127.17.44.253@KRBTEST.COM (short username kudu)
I20260812 11:12:23.622707 6131 webserver.cc:460] Webserver started at http://127.17.44.253:38533/ using document root <none> and password file <none>
I20260812 11:12:23.623350 6131 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:23.623446 6131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:12:23.623684 6131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:12:23.625705 6131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-1/data/instance:
uuid: "582384ed880a42fa989022f89d04aa3a"
format_stamp: "Formatted at 2026-08-12 11:12:23 on dist-test-slave-drl0"
server_key: "73eaf56da9c97200652211a1dc80366a"
server_key_iv: "98906e69c1871e4097dd4e2c650ad9aa"
server_key_version: "encryptionkey@0"
I20260812 11:12:23.626318 6131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-1/wal/instance:
uuid: "582384ed880a42fa989022f89d04aa3a"
format_stamp: "Formatted at 2026-08-12 11:12:23 on dist-test-slave-drl0"
server_key: "73eaf56da9c97200652211a1dc80366a"
server_key_iv: "98906e69c1871e4097dd4e2c650ad9aa"
server_key_version: "encryptionkey@0"
I20260812 11:12:23.630151 6131 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 11:12:23.633464 6158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:23.635125 6131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:12:23.635308 6131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-1/wal
uuid: "582384ed880a42fa989022f89d04aa3a"
format_stamp: "Formatted at 2026-08-12 11:12:23 on dist-test-slave-drl0"
server_key: "73eaf56da9c97200652211a1dc80366a"
server_key_iv: "98906e69c1871e4097dd4e2c650ad9aa"
server_key_version: "encryptionkey@0"
I20260812 11:12:23.635442 6131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:23.660012 6131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:23.663378 6131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:23.663749 6131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:23.671939 6131 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.253:35715
I20260812 11:12:23.672022 6231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.253:35715 every 8 connection(s)
I20260812 11:12:23.672318 6121 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } attempt: 3
I20260812 11:12:23.673130 6131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-1/data/info.pb
I20260812 11:12:23.676448 6232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:12:23.679299 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 6131
I20260812 11:12:23.679440 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-1/wal/instance
I20260812 11:12:23.679766 17587 external_mini_cluster.cc:1428] Setting key 59c0df4783e3582a4f083b8bf6aa1c40
Aug 12 11:12:23 dist-test-slave-drl0 krb5kdc[6002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533143, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for kudu/127.17.44.253@KRBTEST.COM
I20260812 11:12:23.681478 6232 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43701 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:12:23.685981 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.672867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:35715 (local address 127.0.0.1:53756)
0812 11:12:23.673043 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.673062 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:23.673168 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:23.675508 (+ 2340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:23.675511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:23.675712 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:23.675855 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:23.675862 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.677218 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.677225 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:23.677773 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:23.677782 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.678507 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.678511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:23.678620 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:23.679221 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:23.679240 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:23.680883 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:23.683853 (+ 2970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:23.683859 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:23.683861 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:23.684131 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:23.684506 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:23.684512 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:23.684517 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:23.684621 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:23.685431 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:23.685434 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:23.685542 (+ 108us) client_negotiation.cc:769] Sending connection context
0812 11:12:23.685655 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 11:12:23.685790 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260812 11:12:23.687366 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.673597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53756 (local address 127.17.44.253:35715)
0812 11:12:23.674058 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:23.674078 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:23.674128 (+ 50us) server_negotiation.cc:407] Connection header received
0812 11:12:23.674789 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:23.674805 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:23.675230 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:23.675574 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:23.676035 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.677088 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.677905 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.678372 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.678552 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:23.681202 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:23.681208 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:23.681223 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:23.681254 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:23.683725 (+ 2471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.684264 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.684270 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.684278 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.684397 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.684731 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.684734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.684736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.685250 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:23.685367 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:23.686175 (+ 808us) server_negotiation.cc:299] Negotiation successful
0812 11:12:23.686483 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":156,"threads_started":1}
I20260812 11:12:23.689329 6121 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:12:23.690117 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.689800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:42583 (local address 127.0.0.1:53000)
0812 11:12:23.689984 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.690028 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:42583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
WARNING: no policy specified for kudu/127.17.44.252@KRBTEST.COM; defaulting to no policy
I20260812 11:12:23.690972 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.690632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:42583 (local address 127.0.0.1:53010)
0812 11:12:23.690815 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.690872 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:42583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260812 11:12:23.691123 6121 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.252:42583: Network error: Client connection negotiation failed: client connection to 127.17.44.252:42583: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.252@KRBTEST.COM" created.
Aug 12 11:12:23 dist-test-slave-drl0 krb5kdc[6002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533143, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.253@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:12:23.698766 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.684065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36514 (local address 127.17.44.254:43701)
0812 11:12:23.684202 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:23.684206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:23.684912 (+ 706us) server_negotiation.cc:407] Connection header received
0812 11:12:23.685035 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:23.685041 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:23.685107 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:23.685214 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:23.686020 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.686913 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.688128 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.689358 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.689502 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:23.694607 (+ 5105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:23.694611 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:23.694614 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:23.694647 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:23.696387 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.697234 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.697237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.697238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.697322 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.697775 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.697778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.697779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.697972 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:23.698061 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:23.698553 (+ 492us) server_negotiation.cc:299] Negotiation successful
0812 11:12:23.698652 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 11:12:23.699038 6237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.684034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:43701 (local address 127.0.0.1:36514)
0812 11:12:23.684765 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.684822 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:23.684949 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:23.685254 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:23.685260 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:23.685707 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:23.685869 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:23.685877 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.687106 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.687110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:23.687993 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:23.688002 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.689527 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.689531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:23.689650 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:23.690238 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:23.690258 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:23.694364 (+ 4106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:23.696554 (+ 2190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:23.696569 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:23.696579 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:23.696995 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:23.697481 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:23.697488 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:23.697493 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:23.697628 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:23.698096 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:23.698106 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:23.698257 (+ 151us) client_negotiation.cc:769] Sending connection context
0812 11:12:23.698454 (+ 197us) client_negotiation.cc:240] Negotiation successful
0812 11:12:23.698712 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":91,"threads_started":1}
I20260812 11:12:23.700345 6232 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 11:12:23 dist-test-slave-drl0 krb5kdc[6002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533143, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.253@KRBTEST.COM for kudu/127.17.44.253@KRBTEST.COM
I20260812 11:12:23.710954 6237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.700954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:35715 (local address 127.0.0.1:53764)
0812 11:12:23.701134 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.701144 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:23.701213 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:23.701516 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:23.701518 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:23.701665 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:23.701748 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:23.701753 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.702998 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.703002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:23.703510 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:23.703519 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.704166 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.704170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:23.704242 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:23.704801 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:23.704819 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:23.707398 (+ 2579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:23.709343 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:23.709349 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:23.709352 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:23.709544 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:23.709826 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:23.709829 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:23.709831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:23.709908 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:23.710367 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:23.710370 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:23.710464 (+ 94us) client_negotiation.cc:769] Sending connection context
0812 11:12:23.710586 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 11:12:23.710733 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 11:12:23.710980 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.701018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53764 (local address 127.17.44.253:35715)
0812 11:12:23.701111 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:23.701114 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:23.701211 (+ 97us) server_negotiation.cc:407] Connection header received
0812 11:12:23.701309 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:23.701313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:23.701367 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:23.701466 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:23.701872 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.702705 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.703631 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.704018 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.704139 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:23.707541 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:23.707544 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:23.707546 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:23.707567 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:23.709233 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.709644 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.709649 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.709651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.709728 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.710022 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.710027 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.710029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.710273 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:23.710378 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:23.710630 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 11:12:23.710814 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":83,"server-negotiator.queue_time_us":32}
I20260812 11:12:23.712021 6232 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:12:23.712776 6237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.712514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:42583 (local address 127.0.0.1:53016)
0812 11:12:23.712644 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.712717 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:42583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 11:12:23.713361 6160 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.44.252:42583: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.252.keytab.
Entry for principal kudu/127.17.44.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.252.keytab.
I20260812 11:12:23.714351 6237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.714105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:42583 (local address 127.0.0.1:53030)
0812 11:12:23.714230 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.714289 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:42583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 11:12:23.714670 6232 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.252:42583: Network error: Client connection negotiation failed: client connection to 127.17.44.252:42583: connect: Connection refused (error 111)
I20260812 11:12:23.714727 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.252:42583
--webserver_interface=127.17.44.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.252.keytab
--principal=kudu/127.17.44.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:39933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:43701,127.17.44.253:35715,127.17.44.252:42583
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:12:23.722347 6121 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } attempt: 1
I20260812 11:12:23.723047 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.722781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:42583 (local address 127.0.0.1:53046)
0812 11:12:23.722904 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.722984 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:42583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 11:12:23.723758 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.723550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:42583 (local address 127.0.0.1:53048)
0812 11:12:23.723655 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.723708 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:42583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 11:12:23.723922 6121 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.252:42583: Network error: Client connection negotiation failed: client connection to 127.17.44.252:42583: connect: Connection refused (error 111)
I20260812 11:12:23.763990 6232 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } attempt: 1
I20260812 11:12:23.764955 6237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.764598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:42583 (local address 127.0.0.1:53060)
0812 11:12:23.764755 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.764829 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:42583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260812 11:12:23.765770 6237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.765543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:42583 (local address 127.0.0.1:53066)
0812 11:12:23.765672 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.765720 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:42583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260812 11:12:23.766002 6232 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.252:42583: Network error: Client connection negotiation failed: client connection to 127.17.44.252:42583: connect: Connection refused (error 111)
I20260812 11:12:23.799077 6121 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } attempt: 2
I20260812 11:12:23.799937 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.799644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:42583 (local address 127.0.0.1:53082)
0812 11:12:23.799785 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.799852 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:42583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 11:12:23.800822 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.800576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:42583 (local address 127.0.0.1:53086)
0812 11:12:23.800719 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.800768 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:42583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260812 11:12:23.801075 6121 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.252:42583: Network error: Client connection negotiation failed: client connection to 127.17.44.252:42583: connect: Connection refused (error 111)
I20260812 11:12:23.834203 6232 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } attempt: 2
I20260812 11:12:23.835105 6237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.834768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:42583 (local address 127.0.0.1:53090)
0812 11:12:23.834968 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.835029 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:42583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 11:12:23.835858 6237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.835635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:42583 (local address 127.0.0.1:53094)
0812 11:12:23.835746 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.835804 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:42583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 11:12:23.835974 6232 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.252:42583: Network error: Client connection negotiation failed: client connection to 127.17.44.252:42583: connect: Connection refused (error 111)
W20260812 11:12:23.839811 6242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:23.840081 6242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:23.840189 6242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:23.844332 6242 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:12:23.844409 6242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:23.844466 6242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:23.844527 6242 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:12:23.844563 6242 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:12:23.849792 6242 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:39933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:43701,127.17.44.253:35715,127.17.44.252:42583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.252.keytab
--principal=kudu/127.17.44.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.252:42583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.44.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:23.851076 6242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:23.852115 6242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:23.860906 6248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:23.861003 6256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:23.861217 6242 server_base.cc:1034] running on GCE node
W20260812 11:12:23.861261 6250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:23.861831 6242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:23.862898 6242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:23.864251 6242 hybrid_clock.cc:648] HybridClock initialized: now 1786533143864200 us; error 42 us; skew 500 ppm
Aug 12 11:12:23 dist-test-slave-drl0 krb5kdc[6002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533143, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:23.867802 6242 init.cc:377] Logged in from keytab as kudu/127.17.44.252@KRBTEST.COM (short username kudu)
I20260812 11:12:23.868759 6242 webserver.cc:460] Webserver started at http://127.17.44.252:38449/ using document root <none> and password file <none>
I20260812 11:12:23.869300 6242 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:23.869367 6242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:12:23.869537 6242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:12:23.871583 6242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-2/data/instance:
uuid: "cf2ebb9cf7c24db194da55020b8ec208"
format_stamp: "Formatted at 2026-08-12 11:12:23 on dist-test-slave-drl0"
server_key: "59f9382ba5d8a5fb6bc9d54f2b53ce97"
server_key_iv: "7ddd81e84569fe2e033098a19d552f2e"
server_key_version: "encryptionkey@0"
I20260812 11:12:23.872093 6242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-2/wal/instance:
uuid: "cf2ebb9cf7c24db194da55020b8ec208"
format_stamp: "Formatted at 2026-08-12 11:12:23 on dist-test-slave-drl0"
server_key: "59f9382ba5d8a5fb6bc9d54f2b53ce97"
server_key_iv: "7ddd81e84569fe2e033098a19d552f2e"
server_key_version: "encryptionkey@0"
I20260812 11:12:23.876992 6242 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 11:12:23.880102 6272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:23.881668 6242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:12:23.881794 6242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-2/wal
uuid: "cf2ebb9cf7c24db194da55020b8ec208"
format_stamp: "Formatted at 2026-08-12 11:12:23 on dist-test-slave-drl0"
server_key: "59f9382ba5d8a5fb6bc9d54f2b53ce97"
server_key_iv: "7ddd81e84569fe2e033098a19d552f2e"
server_key_version: "encryptionkey@0"
I20260812 11:12:23.881904 6242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:23.896055 6242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:23.899434 6242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:23.899595 6242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:23.900318 6121 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } attempt: 3
I20260812 11:12:23.901132 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.900805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:42583 (local address 127.0.0.1:53104)
0812 11:12:23.900983 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.901055 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:42583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 11:12:23.901981 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.901734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:42583 (local address 127.0.0.1:53108)
0812 11:12:23.901869 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.901917 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:42583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260812 11:12:23.902213 6121 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.252:42583: Network error: Client connection negotiation failed: client connection to 127.17.44.252:42583: connect: Connection refused (error 111)
I20260812 11:12:23.908277 6242 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.252:42583
I20260812 11:12:23.908309 6355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.252:42583 every 8 connection(s)
I20260812 11:12:23.909240 6242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-2/data/info.pb
I20260812 11:12:23.913164 6358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:12:23.913244 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 6242
I20260812 11:12:23.913373 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786532961723938-17587-0/minicluster-data/master-2/wal/instance
I20260812 11:12:23.913820 17587 external_mini_cluster.cc:1428] Setting key 73d312018ff28fd141e3ff650179e4bd
I20260812 11:12:23.918076 6358 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43701 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 11:12:23 dist-test-slave-drl0 krb5kdc[6002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.252@KRBTEST.COM
Aug 12 11:12:23 dist-test-slave-drl0 krb5kdc[6002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533143, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.252@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:12:23.927136 6232 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } attempt: 3
I20260812 11:12:23.930112 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.915376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53114 (local address 127.17.44.252:42583)
0812 11:12:23.915842 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:23.915856 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:23.915892 (+ 36us) server_negotiation.cc:407] Connection header received
0812 11:12:23.916579 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:23.916595 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:23.916897 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:23.917280 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:23.918044 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.919593 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.920903 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.921381 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.921578 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:23.924626 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:23.924635 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:23.924651 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:23.924688 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:23.926892 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.927441 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.927450 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.927458 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.927602 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.927933 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.927936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.927938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.928379 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:23.928551 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:23.928914 (+ 363us) server_negotiation.cc:299] Negotiation successful
0812 11:12:23.929202 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":129,"threads_started":1}
I20260812 11:12:23.931767 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.920070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36524 (local address 127.17.44.254:43701)
0812 11:12:23.920192 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:23.920197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:23.920441 (+ 244us) server_negotiation.cc:407] Connection header received
0812 11:12:23.920573 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:23.920579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:23.920631 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:23.920704 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:23.921423 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.921950 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.923494 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.923899 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.924026 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:23.927294 (+ 3268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:23.927297 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:23.927300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:23.927326 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:23.929284 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.929834 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.929838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.929840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.929940 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.930260 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.930263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.930265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.930519 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:23.930591 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:23.931466 (+ 875us) server_negotiation.cc:299] Negotiation successful
0812 11:12:23.931588 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 11:12:23.932282 6364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.919881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:43701 (local address 127.0.0.1:36524)
0812 11:12:23.920307 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.920356 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:23.920477 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:23.920783 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:23.920791 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:23.921160 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:23.921290 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:23.921296 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.922133 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.922137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:23.923363 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:23.923379 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.924069 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.924073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:23.924185 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:23.924721 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:23.924736 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:23.927138 (+ 2402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:23.929403 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:23.929414 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:23.929423 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:23.929722 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:23.930046 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:23.930049 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:23.930051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:23.930148 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:23.930675 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:23.930681 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:23.931206 (+ 525us) client_negotiation.cc:769] Sending connection context
0812 11:12:23.931895 (+ 689us) client_negotiation.cc:240] Negotiation successful
0812 11:12:23.932070 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":164,"threads_started":1}
I20260812 11:12:23.933705 6358 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 11:12:23 dist-test-slave-drl0 krb5kdc[6002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533143, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.253@KRBTEST.COM for kudu/127.17.44.252@KRBTEST.COM
Aug 12 11:12:23 dist-test-slave-drl0 krb5kdc[6002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:12:23.938663 6237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.928011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:42583 (local address 127.0.0.1:53126)
0812 11:12:23.928603 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.928620 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:23.928725 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:23.929043 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:23.929046 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:23.929243 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:23.929377 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:23.929387 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.930428 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.930431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:23.931642 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:23.931653 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.932250 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.932254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:23.932354 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:23.932847 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:23.932862 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:23.934338 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:23.937058 (+ 2720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:23.937061 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:23.937063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:23.937272 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:23.937560 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:23.937563 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:23.937565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:23.937653 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:23.938094 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:23.938098 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:23.938215 (+ 117us) client_negotiation.cc:769] Sending connection context
0812 11:12:23.938353 (+ 138us) client_negotiation.cc:240] Negotiation successful
0812 11:12:23.938493 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128}
I20260812 11:12:23.939280 6365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.927713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53126 (local address 127.17.44.252:42583)
0812 11:12:23.928120 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:23.928129 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:23.928687 (+ 558us) server_negotiation.cc:407] Connection header received
0812 11:12:23.928846 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:23.928849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:23.928896 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:23.929510 (+ 614us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:23.929602 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.930303 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.931792 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.932097 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.934046 (+ 1949us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:23.934500 (+ 454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:23.934504 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:23.934506 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:23.934530 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:23.936912 (+ 2382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.937373 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.937375 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.937376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.937457 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.937750 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.937755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.937756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.937978 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:23.938101 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:23.938996 (+ 895us) server_negotiation.cc:299] Negotiation successful
0812 11:12:23.939108 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":342,"thread_start_us":119,"threads_started":1}
Aug 12 11:12:23 dist-test-slave-drl0 krb5kdc[6002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533143, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.252@KRBTEST.COM for kudu/127.17.44.253@KRBTEST.COM
I20260812 11:12:23.943967 6364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.934174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:35715 (local address 127.0.0.1:53768)
0812 11:12:23.934275 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.934290 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:23.934503 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:23.934735 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:23.934738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:23.934898 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:23.935048 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:23.935054 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.935951 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.935956 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:23.936931 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:23.936950 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.937716 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.937722 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:23.937822 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:23.938375 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:23.938389 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:23.939747 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:23.942141 (+ 2394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:23.942145 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:23.942147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:23.942344 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:23.942632 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:23.942636 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:23.942638 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:23.942726 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:23.943298 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:23.943301 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:23.943406 (+ 105us) client_negotiation.cc:769] Sending connection context
0812 11:12:23.943673 (+ 267us) client_negotiation.cc:240] Negotiation successful
0812 11:12:23.943807 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":27}
I20260812 11:12:23.943941 6232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a: Bootstrap starting.
I20260812 11:12:23.944284 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.932496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36536 (local address 127.17.44.254:43701)
0812 11:12:23.932616 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:23.932621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:23.932638 (+ 17us) server_negotiation.cc:407] Connection header received
0812 11:12:23.932707 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:23.932722 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:23.932767 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:23.932852 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:23.933811 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.934515 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.935756 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.936295 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.936424 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:23.939169 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:23.939172 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:23.939175 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:23.939198 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:23.941438 (+ 2240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.941940 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.941944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.941945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.942032 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.942350 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.942353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.942354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.942582 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:23.943722 (+ 1140us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:23.943894 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 11:12:23.944102 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 11:12:23.944422 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.934413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53768 (local address 127.17.44.253:35715)
0812 11:12:23.934510 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:23.934514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:23.934529 (+ 15us) server_negotiation.cc:407] Connection header received
0812 11:12:23.934592 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:23.934595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:23.934639 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:23.934721 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:23.935156 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.935831 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.937060 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.937457 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.937577 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:23.940144 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:23.940147 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:23.940150 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:23.940176 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:23.942036 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.942449 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.942452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.942454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.942537 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.942820 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.942825 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.942827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.943149 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:23.943250 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:23.943741 (+ 491us) server_negotiation.cc:299] Negotiation successful
0812 11:12:23.943845 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 11:12:23.945873 6358 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:12:23.949571 6232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a: Neither blocks nor log segments found. Creating new log.
I20260812 11:12:23.950531 6232 log.cc:824] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a: Log is configured to *not* fsync() on all Append() calls
Aug 12 11:12:23 dist-test-slave-drl0 krb5kdc[6002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.253@KRBTEST.COM
I20260812 11:12:23.953938 6232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a: No bootstrap required, opened a new log
I20260812 11:12:23.955423 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.945943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53772 (local address 127.17.44.253:35715)
0812 11:12:23.946081 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:23.946085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:23.946136 (+ 51us) server_negotiation.cc:407] Connection header received
0812 11:12:23.946181 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:23.946184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:23.946237 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:23.946328 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:23.947195 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.947990 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.948911 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.949224 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.949360 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:23.951909 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:23.951912 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:23.951915 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:23.951939 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:23.953528 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.954030 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.954034 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.954036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.954116 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.954421 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.954425 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.954426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.954632 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:23.954826 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:23.955135 (+ 309us) server_negotiation.cc:299] Negotiation successful
0812 11:12:23.955269 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 12 11:12:23 dist-test-slave-drl0 krb5kdc[6002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533143, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.252@KRBTEST.COM for kudu/127.17.44.252@KRBTEST.COM
I20260812 11:12:23.958698 6232 raft_consensus.cc:348] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b0e89a66294cdb82ade6b8efc2a92a" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43701 } } peers { permanent_uuid: "582384ed880a42fa989022f89d04aa3a" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } } peers { permanent_uuid: "cf2ebb9cf7c24db194da55020b8ec208" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } }
I20260812 11:12:23.958992 6232 raft_consensus.cc:374] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:12:23.959057 6232 raft_consensus.cc:729] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 582384ed880a42fa989022f89d04aa3a, State: Initialized, Role: FOLLOWER
I20260812 11:12:23.959620 6232 consensus_queue.cc:260] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b0e89a66294cdb82ade6b8efc2a92a" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43701 } } peers { permanent_uuid: "582384ed880a42fa989022f89d04aa3a" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } } peers { permanent_uuid: "cf2ebb9cf7c24db194da55020b8ec208" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } }
I20260812 11:12:23.960210 6367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b0e89a66294cdb82ade6b8efc2a92a" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43701 } } peers { permanent_uuid: "582384ed880a42fa989022f89d04aa3a" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } } peers { permanent_uuid: "cf2ebb9cf7c24db194da55020b8ec208" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } } }
I20260812 11:12:23.960453 6367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:12:23.960595 6364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.946594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:42583 (local address 127.0.0.1:53132)
0812 11:12:23.946798 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:23.946814 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:23.946896 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:23.951599 (+ 4703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:23.951602 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:23.951892 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:23.952019 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:23.952029 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.953096 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.953099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:23.953624 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:23.953636 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.954469 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.954471 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:23.954523 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:23.955058 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:23.955073 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:23.956448 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:23.959101 (+ 2653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:23.959103 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:23.959106 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:23.959337 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:23.959602 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:23.959604 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:23.959606 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:23.959678 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:23.960169 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:23.960171 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:23.960245 (+ 74us) client_negotiation.cc:769] Sending connection context
0812 11:12:23.960333 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 11:12:23.960453 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260812 11:12:23.960992 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:12:23.961139 6365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.946703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53132 (local address 127.17.44.252:42583)
0812 11:12:23.951225 (+ 4522us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:23.951231 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:23.951251 (+ 20us) server_negotiation.cc:407] Connection header received
0812 11:12:23.951304 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:23.951310 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:23.951365 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:23.951464 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:23.952230 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.952962 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.953756 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.954225 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.954341 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:23.956587 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:23.956590 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:23.956593 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:23.956615 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:23.958979 (+ 2364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.959441 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.959444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.959445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.959516 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.959767 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.959772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.959774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.960044 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:23.960138 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:23.960883 (+ 745us) server_negotiation.cc:299] Negotiation successful
0812 11:12:23.961016 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4407}
I20260812 11:12:23.961187 6232 sys_catalog.cc:564] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:12:23.964135 6358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf2ebb9cf7c24db194da55020b8ec208: Bootstrap starting.
I20260812 11:12:23.967219 6358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf2ebb9cf7c24db194da55020b8ec208: Neither blocks nor log segments found. Creating new log.
I20260812 11:12:23.968201 6358 log.cc:824] T 00000000000000000000000000000000 P cf2ebb9cf7c24db194da55020b8ec208: Log is configured to *not* fsync() on all Append() calls
I20260812 11:12:23.971596 6358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf2ebb9cf7c24db194da55020b8ec208: No bootstrap required, opened a new log
I20260812 11:12:23.973019 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.963754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36552 (local address 127.17.44.254:43701)
0812 11:12:23.963892 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:23.963897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:23.963910 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:12:23.963965 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:23.963968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:23.964018 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:23.964090 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:23.965072 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.965783 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.967049 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.967315 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.967413 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:23.969160 (+ 1747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:23.969163 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:23.969166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:23.969196 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:23.970987 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.971495 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.971498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.971500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.971582 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.971872 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.971877 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.971881 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.972089 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:23.972208 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:23.972751 (+ 543us) server_negotiation.cc:299] Negotiation successful
0812 11:12:23.972866 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260812 11:12:23.974320 6392 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:12:23.974399 6392 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:12:23.976845 6358 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf2ebb9cf7c24db194da55020b8ec208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b0e89a66294cdb82ade6b8efc2a92a" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43701 } } peers { permanent_uuid: "582384ed880a42fa989022f89d04aa3a" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } } peers { permanent_uuid: "cf2ebb9cf7c24db194da55020b8ec208" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } }
I20260812 11:12:23.972981 6131 master_runner.cc:186] Error getting master registration for 127.17.44.254:43701: OK, instance_id { permanent_uuid: "e1b0e89a66294cdb82ade6b8efc2a92a" instance_seqno: 1786533143426241 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 11:12:23.977172 6131 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 11:12:23.977201 6358 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf2ebb9cf7c24db194da55020b8ec208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:12:23.977388 6358 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf2ebb9cf7c24db194da55020b8ec208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf2ebb9cf7c24db194da55020b8ec208, State: Initialized, Role: FOLLOWER
I20260812 11:12:23.978185 6358 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf2ebb9cf7c24db194da55020b8ec208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b0e89a66294cdb82ade6b8efc2a92a" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43701 } } peers { permanent_uuid: "582384ed880a42fa989022f89d04aa3a" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } } peers { permanent_uuid: "cf2ebb9cf7c24db194da55020b8ec208" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } }
I20260812 11:12:23.979017 6394 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf2ebb9cf7c24db194da55020b8ec208 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b0e89a66294cdb82ade6b8efc2a92a" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43701 } } peers { permanent_uuid: "582384ed880a42fa989022f89d04aa3a" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } } peers { permanent_uuid: "cf2ebb9cf7c24db194da55020b8ec208" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } } }
I20260812 11:12:23.979166 6394 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf2ebb9cf7c24db194da55020b8ec208 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:12:23.980026 6358 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf2ebb9cf7c24db194da55020b8ec208 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:12:23.981608 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.963531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53788 (local address 127.17.44.253:35715)
0812 11:12:23.963652 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:23.963656 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:23.964479 (+ 823us) server_negotiation.cc:407] Connection header received
0812 11:12:23.964563 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:23.964568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:23.964624 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:23.964719 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:23.965519 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.966124 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.967508 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.967740 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.967855 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:23.969560 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:23.969566 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:23.969571 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:23.969599 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:23.972749 (+ 3150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.979295 (+ 6546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.979302 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.979304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.979455 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.980031 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.980036 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.980037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.980443 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:23.980609 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:23.981306 (+ 697us) server_negotiation.cc:299] Negotiation successful
0812 11:12:23.981440 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 11:12:23.981699 6365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:23.964088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53146 (local address 127.17.44.252:42583)
0812 11:12:23.964200 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:23.964204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:23.970692 (+ 6488us) server_negotiation.cc:407] Connection header received
0812 11:12:23.970872 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:23.970876 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:23.970947 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:23.971003 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:23.973750 (+ 2747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.974446 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.975559 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:23.975907 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:23.975978 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:23.977584 (+ 1606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:23.977588 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:23.977591 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:23.977618 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:23.979601 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.980228 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.980232 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.980233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.980317 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:23.980644 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:23.980647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:23.980649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:23.980877 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:23.981038 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:23.981435 (+ 397us) server_negotiation.cc:299] Negotiation successful
0812 11:12:23.981545 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260812 11:12:23.986521 6415 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cf2ebb9cf7c24db194da55020b8ec208: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:12:23.986601 6415 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:12:23.985013 6242 master_runner.cc:186] Error getting master registration for 127.17.44.254:43701: OK, instance_id { permanent_uuid: "e1b0e89a66294cdb82ade6b8efc2a92a" instance_seqno: 1786533143426241 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 11:12:23.988363 6242 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 11:12:23.997143 6367 raft_consensus.cc:482] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:12:23.997339 6367 raft_consensus.cc:504] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b0e89a66294cdb82ade6b8efc2a92a" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43701 } } peers { permanent_uuid: "582384ed880a42fa989022f89d04aa3a" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } } peers { permanent_uuid: "cf2ebb9cf7c24db194da55020b8ec208" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } }
I20260812 11:12:23.999008 6367 leader_election.cc:288] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1b0e89a66294cdb82ade6b8efc2a92a (127.17.44.254:43701), cf2ebb9cf7c24db194da55020b8ec208 (127.17.44.252:42583)
I20260812 11:12:23.999698 6312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "582384ed880a42fa989022f89d04aa3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf2ebb9cf7c24db194da55020b8ec208" is_pre_election: true
I20260812 11:12:24.000083 6312 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cf2ebb9cf7c24db194da55020b8ec208 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 582384ed880a42fa989022f89d04aa3a in term 0.
I20260812 11:12:24.000614 6160 leader_election.cc:302] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 582384ed880a42fa989022f89d04aa3a, cf2ebb9cf7c24db194da55020b8ec208; no voters:
I20260812 11:12:24.000875 6367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:12:24.000993 6367 raft_consensus.cc:482] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:12:24.001050 6367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:12:24.003347 6367 raft_consensus.cc:504] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b0e89a66294cdb82ade6b8efc2a92a" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43701 } } peers { permanent_uuid: "582384ed880a42fa989022f89d04aa3a" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } } peers { permanent_uuid: "cf2ebb9cf7c24db194da55020b8ec208" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } }
I20260812 11:12:24.004173 6367 leader_election.cc:288] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [CANDIDATE]: Term 1 election: Requested vote from peers e1b0e89a66294cdb82ade6b8efc2a92a (127.17.44.254:43701), cf2ebb9cf7c24db194da55020b8ec208 (127.17.44.252:42583)
I20260812 11:12:24.005110 6312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "582384ed880a42fa989022f89d04aa3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf2ebb9cf7c24db194da55020b8ec208"
I20260812 11:12:24.005242 6312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf2ebb9cf7c24db194da55020b8ec208 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:12:24.006623 6312 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cf2ebb9cf7c24db194da55020b8ec208 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 582384ed880a42fa989022f89d04aa3a in term 1.
I20260812 11:12:24.007602 6160 leader_election.cc:302] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 582384ed880a42fa989022f89d04aa3a, cf2ebb9cf7c24db194da55020b8ec208; no voters:
I20260812 11:12:24.008044 6367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:12:24.008283 6367 raft_consensus.cc:686] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [term 1 LEADER]: Becoming Leader. State: Replica: 582384ed880a42fa989022f89d04aa3a, State: Running, Role: LEADER
I20260812 11:12:24.008643 6367 consensus_queue.cc:237] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b0e89a66294cdb82ade6b8efc2a92a" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43701 } } peers { permanent_uuid: "582384ed880a42fa989022f89d04aa3a" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } } peers { permanent_uuid: "cf2ebb9cf7c24db194da55020b8ec208" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } }
I20260812 11:12:24.010427 6367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 582384ed880a42fa989022f89d04aa3a. Latest consensus state: current_term: 1 leader_uuid: "582384ed880a42fa989022f89d04aa3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b0e89a66294cdb82ade6b8efc2a92a" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43701 } } peers { permanent_uuid: "582384ed880a42fa989022f89d04aa3a" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } } peers { permanent_uuid: "cf2ebb9cf7c24db194da55020b8ec208" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } } }
I20260812 11:12:24.010537 6367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [sys.catalog]: This master's current role is: LEADER
I20260812 11:12:24.012897 6420 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:12:24.013896 6420 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:12:24.019333 6312 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cf2ebb9cf7c24db194da55020b8ec208 [term 1 FOLLOWER]: Refusing update from remote peer 582384ed880a42fa989022f89d04aa3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:12:24.021289 6417 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2ebb9cf7c24db194da55020b8ec208" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 11:12:24.027884 6394 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf2ebb9cf7c24db194da55020b8ec208 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 582384ed880a42fa989022f89d04aa3a. Latest consensus state: current_term: 1 leader_uuid: "582384ed880a42fa989022f89d04aa3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b0e89a66294cdb82ade6b8efc2a92a" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43701 } } peers { permanent_uuid: "582384ed880a42fa989022f89d04aa3a" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } } peers { permanent_uuid: "cf2ebb9cf7c24db194da55020b8ec208" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } } }
I20260812 11:12:24.028010 6394 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf2ebb9cf7c24db194da55020b8ec208 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:12:24.031867 6417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "582384ed880a42fa989022f89d04aa3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b0e89a66294cdb82ade6b8efc2a92a" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43701 } } peers { permanent_uuid: "582384ed880a42fa989022f89d04aa3a" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } } peers { permanent_uuid: "cf2ebb9cf7c24db194da55020b8ec208" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } } }
I20260812 11:12:24.032003 6417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [sys.catalog]: This master's current role is: LEADER
I20260812 11:12:24.033730 6394 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf2ebb9cf7c24db194da55020b8ec208 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "582384ed880a42fa989022f89d04aa3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b0e89a66294cdb82ade6b8efc2a92a" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43701 } } peers { permanent_uuid: "582384ed880a42fa989022f89d04aa3a" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } } peers { permanent_uuid: "cf2ebb9cf7c24db194da55020b8ec208" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } } }
I20260812 11:12:24.033845 6394 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf2ebb9cf7c24db194da55020b8ec208 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:12:24.034987 6424 mvcc.cc:205] Tried to move back new op lower bound from 7317639757896454144 to 7317639757862539264. Current Snapshot: MvccSnapshot[applied={T|T < 7317639757896454144}]
I20260812 11:12:24.036517 6420 catalog_manager.cc:1347] Generated new cluster ID: f78f2f28493c4ca7a533cf180c769c88
I20260812 11:12:24.036574 6420 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:12:24.053334 6420 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:12:24.054780 6420 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:12:24.066397 6121 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } attempt: 4
I20260812 11:12:24.068194 6420 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a: Generated new TSK 0
I20260812 11:12:24.070961 6420 catalog_manager.cc:1514] Initializing in-progress tserver states...
Aug 12 11:12:24 dist-test-slave-drl0 krb5kdc[6002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533143, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for kudu/127.17.44.252@KRBTEST.COM
I20260812 11:12:24.081414 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:24.066824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:42583 (local address 127.0.0.1:53150)
0812 11:12:24.066963 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:24.066980 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:24.067067 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:24.067634 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:24.067637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:24.067864 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:24.067971 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:24.067977 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:24.068963 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:24.068970 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:24.069525 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:24.069561 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:24.070236 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:24.070240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:24.070328 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:24.074707 (+ 4379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:24.074723 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:24.077028 (+ 2305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:24.079537 (+ 2509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:24.079543 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:24.079548 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:24.079831 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:24.080183 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:24.080189 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:24.080194 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:24.080315 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:24.080821 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:24.080825 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:24.080935 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 11:12:24.081063 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 11:12:24.081220 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 11:12:24.082384 6365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:24.067104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53150 (local address 127.17.44.252:42583)
0812 11:12:24.067215 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:24.067221 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:24.067233 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:12:24.067269 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:24.067271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:24.067321 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:24.067402 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:24.068096 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:24.068849 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:24.069667 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:24.070002 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:24.070108 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:24.077183 (+ 7075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:24.077189 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:24.077193 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:24.077223 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:24.079398 (+ 2175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:24.079958 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:24.079962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:24.079963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:24.080067 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:24.080429 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:24.080435 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:24.080436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:24.080669 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:24.080769 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:24.082064 (+ 1295us) server_negotiation.cc:299] Negotiation successful
0812 11:12:24.082206 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 11:12:24.086161 6121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1b0e89a66294cdb82ade6b8efc2a92a: Bootstrap starting.
I20260812 11:12:24.089349 6121 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1b0e89a66294cdb82ade6b8efc2a92a: Neither blocks nor log segments found. Creating new log.
I20260812 11:12:24.090317 6121 log.cc:824] T 00000000000000000000000000000000 P e1b0e89a66294cdb82ade6b8efc2a92a: Log is configured to *not* fsync() on all Append() calls
I20260812 11:12:24.093326 6121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1b0e89a66294cdb82ade6b8efc2a92a: No bootstrap required, opened a new log
I20260812 11:12:24.098320 6121 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1b0e89a66294cdb82ade6b8efc2a92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b0e89a66294cdb82ade6b8efc2a92a" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43701 } } peers { permanent_uuid: "582384ed880a42fa989022f89d04aa3a" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } } peers { permanent_uuid: "cf2ebb9cf7c24db194da55020b8ec208" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } }
I20260812 11:12:24.098563 6121 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1b0e89a66294cdb82ade6b8efc2a92a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:12:24.098616 6121 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1b0e89a66294cdb82ade6b8efc2a92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1b0e89a66294cdb82ade6b8efc2a92a, State: Initialized, Role: FOLLOWER
I20260812 11:12:24.099710 6121 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1b0e89a66294cdb82ade6b8efc2a92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b0e89a66294cdb82ade6b8efc2a92a" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43701 } } peers { permanent_uuid: "582384ed880a42fa989022f89d04aa3a" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } } peers { permanent_uuid: "cf2ebb9cf7c24db194da55020b8ec208" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } }
I20260812 11:12:24.101714 6121 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1b0e89a66294cdb82ade6b8efc2a92a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:12:24.102254 6078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "582384ed880a42fa989022f89d04aa3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1b0e89a66294cdb82ade6b8efc2a92a"
I20260812 11:12:24.102607 6078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1b0e89a66294cdb82ade6b8efc2a92a [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:12:24.103288 6080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "582384ed880a42fa989022f89d04aa3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1b0e89a66294cdb82ade6b8efc2a92a" is_pre_election: true
I20260812 11:12:24.105834 6078 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e1b0e89a66294cdb82ade6b8efc2a92a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 582384ed880a42fa989022f89d04aa3a in term 1.
I20260812 11:12:24.106346 6076 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e1b0e89a66294cdb82ade6b8efc2a92a [term 1 FOLLOWER]: Refusing update from remote peer 582384ed880a42fa989022f89d04aa3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:12:24.107295 6367 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1b0e89a66294cdb82ade6b8efc2a92a" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:12:24.107585 6436 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1b0e89a66294cdb82ade6b8efc2a92a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b0e89a66294cdb82ade6b8efc2a92a" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43701 } } peers { permanent_uuid: "582384ed880a42fa989022f89d04aa3a" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } } peers { permanent_uuid: "cf2ebb9cf7c24db194da55020b8ec208" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } } }
I20260812 11:12:24.108021 6436 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1b0e89a66294cdb82ade6b8efc2a92a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:12:24.115020 6436 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1b0e89a66294cdb82ade6b8efc2a92a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "582384ed880a42fa989022f89d04aa3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b0e89a66294cdb82ade6b8efc2a92a" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43701 } } peers { permanent_uuid: "582384ed880a42fa989022f89d04aa3a" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } } peers { permanent_uuid: "cf2ebb9cf7c24db194da55020b8ec208" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } } }
I20260812 11:12:24.115137 6436 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1b0e89a66294cdb82ade6b8efc2a92a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:12:24.117434 6452 mvcc.cc:205] Tried to move back new op lower bound from 7317639758086524928 to 7317639757862539264. Current Snapshot: MvccSnapshot[applied={T|T < 7317639757896454144}]
I20260812 11:12:24.114876 6455 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1b0e89a66294cdb82ade6b8efc2a92a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 582384ed880a42fa989022f89d04aa3a. Latest consensus state: current_term: 1 leader_uuid: "582384ed880a42fa989022f89d04aa3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b0e89a66294cdb82ade6b8efc2a92a" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43701 } } peers { permanent_uuid: "582384ed880a42fa989022f89d04aa3a" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } } peers { permanent_uuid: "cf2ebb9cf7c24db194da55020b8ec208" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } } }
I20260812 11:12:24.118077 6455 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1b0e89a66294cdb82ade6b8efc2a92a [sys.catalog]: This master's current role is: FOLLOWER
W20260812 11:12:24.118278 6451 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e1b0e89a66294cdb82ade6b8efc2a92a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:12:24.118448 6451 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:12:24.710554 6462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "582384ed880a42fa989022f89d04aa3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b0e89a66294cdb82ade6b8efc2a92a" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43701 } } peers { permanent_uuid: "582384ed880a42fa989022f89d04aa3a" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 35715 } } peers { permanent_uuid: "cf2ebb9cf7c24db194da55020b8ec208" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 42583 } } }
I20260812 11:12:24.710783 6462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 582384ed880a42fa989022f89d04aa3a [sys.catalog]: This master's current role is: LEADER
I20260812 11:12:24.988452 6415 catalog_manager.cc:1259] Loaded cluster ID: f78f2f28493c4ca7a533cf180c769c88
I20260812 11:12:24.988543 6415 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cf2ebb9cf7c24db194da55020b8ec208: loading cluster ID for follower catalog manager: success
I20260812 11:12:24.990203 6415 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cf2ebb9cf7c24db194da55020b8ec208: acquiring CA information for follower catalog manager: success
I20260812 11:12:24.991523 6415 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cf2ebb9cf7c24db194da55020b8ec208: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 11:12:25.119813 6451 catalog_manager.cc:1259] Loaded cluster ID: f78f2f28493c4ca7a533cf180c769c88
I20260812 11:12:25.119905 6451 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e1b0e89a66294cdb82ade6b8efc2a92a: loading cluster ID for follower catalog manager: success
I20260812 11:12:25.121616 6451 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e1b0e89a66294cdb82ade6b8efc2a92a: acquiring CA information for follower catalog manager: success
I20260812 11:12:25.122764 6451 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e1b0e89a66294cdb82ade6b8efc2a92a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 11:12:25.147604 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 6019
I20260812 11:12:25.160094 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 6131
I20260812 11:12:25.173092 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 6242
2026-08-12T11:12:25Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3381 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:12:25 dist-test-slave-drl0 krb5kdc[6469](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:12:25 dist-test-slave-drl0 krb5kdc[6469](info): set up 2 sockets
Aug 12 11:12:25 dist-test-slave-drl0 krb5kdc[6469](info): commencing operation
krb5kdc: starting...
W20260812 11:12:26.466177 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:12:26 dist-test-slave-drl0 krb5kdc[6469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:12:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:12:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:12:26.607299 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:44939
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:37871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:44939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:12:26.739923 6493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:26.740345 6493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:26.740437 6493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:26.745591 6493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:12:26.745739 6493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:26.745791 6493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:26.745865 6493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:12:26.745909 6493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:12:26.752818 6493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:37871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:44939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:44939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:26.754473 6493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:26.755879 6493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:26.766979 6500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:26.766975 6501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:26.766973 6503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:26.767040 6493 server_base.cc:1034] running on GCE node
I20260812 11:12:26.768282 6493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:26.769876 6493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:26.771232 6493 hybrid_clock.cc:648] HybridClock initialized: now 1786533146771174 us; error 86 us; skew 500 ppm
Aug 12 11:12:26 dist-test-slave-drl0 krb5kdc[6469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533146, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:26.778919 6493 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:12:26.781229 6493 webserver.cc:460] Webserver started at http://127.17.44.254:33735/ using document root <none> and password file <none>
I20260812 11:12:26.782009 6493 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:26.782125 6493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:12:26.782410 6493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:12:26.784896 6493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "6311667ee450438589e9c43d19399809"
format_stamp: "Formatted at 2026-08-12 11:12:26 on dist-test-slave-drl0"
server_key: "e7d5075c2171e2d5f5c0708a095fb5bc"
server_key_iv: "1675cd172064d58042d2019a578474ad"
server_key_version: "encryptionkey@0"
I20260812 11:12:26.785533 6493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "6311667ee450438589e9c43d19399809"
format_stamp: "Formatted at 2026-08-12 11:12:26 on dist-test-slave-drl0"
server_key: "e7d5075c2171e2d5f5c0708a095fb5bc"
server_key_iv: "1675cd172064d58042d2019a578474ad"
server_key_version: "encryptionkey@0"
I20260812 11:12:26.789857 6493 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 11:12:26.793103 6512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:26.795101 6493 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 11:12:26.795305 6493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "6311667ee450438589e9c43d19399809"
format_stamp: "Formatted at 2026-08-12 11:12:26 on dist-test-slave-drl0"
server_key: "e7d5075c2171e2d5f5c0708a095fb5bc"
server_key_iv: "1675cd172064d58042d2019a578474ad"
server_key_version: "encryptionkey@0"
I20260812 11:12:26.795446 6493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:26.811125 6493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:26.814699 6493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:26.814981 6493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:26.823969 6493 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:44939
I20260812 11:12:26.824290 6596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:44939 every 8 connection(s)
I20260812 11:12:26.825090 6493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:12:26.828094 6598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:12:26.828459 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 6493
I20260812 11:12:26.828650 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:12:26.829077 17587 external_mini_cluster.cc:1428] Setting key cdff2d760b5bc8ffdfea5aa023759f96
I20260812 11:12:26.835575 6598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6311667ee450438589e9c43d19399809: Bootstrap starting.
I20260812 11:12:26.839608 6598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6311667ee450438589e9c43d19399809: Neither blocks nor log segments found. Creating new log.
I20260812 11:12:26.840773 6598 log.cc:824] T 00000000000000000000000000000000 P 6311667ee450438589e9c43d19399809: Log is configured to *not* fsync() on all Append() calls
Aug 12 11:12:26 dist-test-slave-drl0 krb5kdc[6469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:12:26.843422 6598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6311667ee450438589e9c43d19399809: No bootstrap required, opened a new log
I20260812 11:12:26.847877 6602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:26.831039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35400 (local address 127.17.44.254:44939)
0812 11:12:26.831660 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:26.831682 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:26.831735 (+ 53us) server_negotiation.cc:407] Connection header received
0812 11:12:26.832624 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:26.832649 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:26.833100 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:26.833485 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:26.835066 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:26.836208 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:26.837676 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:26.838268 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:26.838430 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:26.842309 (+ 3879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:26.842317 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:26.842335 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:26.842369 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:26.844418 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:26.845112 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:26.845126 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:26.845131 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:26.845278 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:26.845644 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:26.845647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:26.845649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:26.846118 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:26.846271 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:26.846658 (+ 387us) server_negotiation.cc:299] Negotiation successful
0812 11:12:26.847060 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":194,"threads_started":1}
I20260812 11:12:26.847914 6598 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6311667ee450438589e9c43d19399809 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6311667ee450438589e9c43d19399809" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44939 } }
I20260812 11:12:26.848263 6598 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6311667ee450438589e9c43d19399809 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:12:26.848326 6598 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6311667ee450438589e9c43d19399809 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6311667ee450438589e9c43d19399809, State: Initialized, Role: FOLLOWER
I20260812 11:12:26.848816 6598 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6311667ee450438589e9c43d19399809 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6311667ee450438589e9c43d19399809" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44939 } }
I20260812 11:12:26.848994 6598 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6311667ee450438589e9c43d19399809 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:12:26.849093 6598 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6311667ee450438589e9c43d19399809 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:12:26.849229 6598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6311667ee450438589e9c43d19399809 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:12:26.850392 6598 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6311667ee450438589e9c43d19399809 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6311667ee450438589e9c43d19399809" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44939 } }
I20260812 11:12:26.850756 6598 leader_election.cc:302] T 00000000000000000000000000000000 P 6311667ee450438589e9c43d19399809 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6311667ee450438589e9c43d19399809; no voters:
I20260812 11:12:26.851105 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:12:26.851173 6598 leader_election.cc:288] T 00000000000000000000000000000000 P 6311667ee450438589e9c43d19399809 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:12:26.851254 6604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6311667ee450438589e9c43d19399809 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:12:26.851459 6604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6311667ee450438589e9c43d19399809 [term 1 LEADER]: Becoming Leader. State: Replica: 6311667ee450438589e9c43d19399809, State: Running, Role: LEADER
I20260812 11:12:26.851830 6604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6311667ee450438589e9c43d19399809 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6311667ee450438589e9c43d19399809" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44939 } }
I20260812 11:12:26.852558 6598 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6311667ee450438589e9c43d19399809 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:12:26.853046 6604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6311667ee450438589e9c43d19399809 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6311667ee450438589e9c43d19399809" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6311667ee450438589e9c43d19399809" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44939 } } }
I20260812 11:12:26.853215 6604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6311667ee450438589e9c43d19399809 [sys.catalog]: This master's current role is: LEADER
I20260812 11:12:26.853477 6605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6311667ee450438589e9c43d19399809 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6311667ee450438589e9c43d19399809. Latest consensus state: current_term: 1 leader_uuid: "6311667ee450438589e9c43d19399809" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6311667ee450438589e9c43d19399809" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44939 } } }
I20260812 11:12:26.853643 6605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6311667ee450438589e9c43d19399809 [sys.catalog]: This master's current role is: LEADER
I20260812 11:12:26.853680 6615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:12:26.857797 6615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:12:26.865819 6615 catalog_manager.cc:1347] Generated new cluster ID: b9b6e00db02645a69868ef4fd7f6c0de
I20260812 11:12:26.865955 6615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:12:26.880509 6615 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:12:26.883348 6615 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:12:26.893766 6615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6311667ee450438589e9c43d19399809: Generated new TSK 0
I20260812 11:12:26.894716 6615 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:12:26.908031 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 6493
2026-08-12T11:12:26Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1739 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:12:26 dist-test-slave-drl0 krb5kdc[6636](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:12:26 dist-test-slave-drl0 krb5kdc[6636](info): set up 2 sockets
Aug 12 11:12:26 dist-test-slave-drl0 krb5kdc[6636](info): commencing operation
krb5kdc: starting...
W20260812 11:12:28.214021 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:12:28 dist-test-slave-drl0 krb5kdc[6636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:12:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:12:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:12:28.350773 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:36829
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:46029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:36829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:12:28.476220 6655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:28.476490 6655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:28.476531 6655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:28.480888 6655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:12:28.481599 6655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:28.481668 6655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:28.481696 6655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:12:28.481714 6655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:12:28.488114 6655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:46029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:36829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:36829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:28.489197 6655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:28.490254 6655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:28.499136 6671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:28.499174 6668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:28.499171 6667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:28.499230 6655 server_base.cc:1034] running on GCE node
I20260812 11:12:28.499981 6655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:28.501067 6655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:28.502249 6655 hybrid_clock.cc:648] HybridClock initialized: now 1786533148502217 us; error 45 us; skew 500 ppm
Aug 12 11:12:28 dist-test-slave-drl0 krb5kdc[6636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533148, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:28.505383 6655 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:12:28.506232 6655 webserver.cc:460] Webserver started at http://127.17.44.254:43893/ using document root <none> and password file <none>
I20260812 11:12:28.506969 6655 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:28.507081 6655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:12:28.507325 6655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:12:28.509415 6655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "b11244d9a967487ba96bdcf19d8b68eb"
format_stamp: "Formatted at 2026-08-12 11:12:28 on dist-test-slave-drl0"
server_key: "94d5f5d03ebfa615a0832c5d8962236d"
server_key_iv: "a6015e6f897679ff3d48f6bbb7b0bac7"
server_key_version: "encryptionkey@0"
I20260812 11:12:28.510083 6655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "b11244d9a967487ba96bdcf19d8b68eb"
format_stamp: "Formatted at 2026-08-12 11:12:28 on dist-test-slave-drl0"
server_key: "94d5f5d03ebfa615a0832c5d8962236d"
server_key_iv: "a6015e6f897679ff3d48f6bbb7b0bac7"
server_key_version: "encryptionkey@0"
I20260812 11:12:28.514357 6655 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260812 11:12:28.517365 6681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:28.518775 6655 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:12:28.519001 6655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "b11244d9a967487ba96bdcf19d8b68eb"
format_stamp: "Formatted at 2026-08-12 11:12:28 on dist-test-slave-drl0"
server_key: "94d5f5d03ebfa615a0832c5d8962236d"
server_key_iv: "a6015e6f897679ff3d48f6bbb7b0bac7"
server_key_version: "encryptionkey@0"
I20260812 11:12:28.519126 6655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:28.542981 6655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:28.547868 6655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:28.548242 6655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:28.556878 6655 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:36829
I20260812 11:12:28.557245 6768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:36829 every 8 connection(s)
I20260812 11:12:28.557941 6655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:12:28.561462 6770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:12:28.567337 6770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb: Bootstrap starting.
I20260812 11:12:28.568224 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 6655
I20260812 11:12:28.568378 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:12:28.568730 17587 external_mini_cluster.cc:1428] Setting key beffdffa14958c3f8aa90677a3480947
I20260812 11:12:28.570333 6770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb: Neither blocks nor log segments found. Creating new log.
I20260812 11:12:28.571151 6770 log.cc:824] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb: Log is configured to *not* fsync() on all Append() calls
I20260812 11:12:28.572965 6770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb: No bootstrap required, opened a new log
I20260812 11:12:28.576429 6770 raft_consensus.cc:348] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b11244d9a967487ba96bdcf19d8b68eb" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36829 } }
I20260812 11:12:28.577047 6770 raft_consensus.cc:374] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:12:28.577178 6770 raft_consensus.cc:729] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b11244d9a967487ba96bdcf19d8b68eb, State: Initialized, Role: FOLLOWER
I20260812 11:12:28.577864 6770 consensus_queue.cc:260] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b11244d9a967487ba96bdcf19d8b68eb" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36829 } }
I20260812 11:12:28.578074 6770 raft_consensus.cc:388] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Aug 12 11:12:28 dist-test-slave-drl0 krb5kdc[6636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:12:28.578166 6770 raft_consensus.cc:482] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:12:28.578325 6770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:12:28.579698 6770 raft_consensus.cc:504] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b11244d9a967487ba96bdcf19d8b68eb" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36829 } }
I20260812 11:12:28.580506 6770 leader_election.cc:302] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b11244d9a967487ba96bdcf19d8b68eb; no voters:
I20260812 11:12:28.581175 6770 leader_election.cc:288] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:12:28.581259 6775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:12:28.581477 6775 raft_consensus.cc:686] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 1 LEADER]: Becoming Leader. State: Replica: b11244d9a967487ba96bdcf19d8b68eb, State: Running, Role: LEADER
I20260812 11:12:28.581956 6775 consensus_queue.cc:237] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b11244d9a967487ba96bdcf19d8b68eb" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36829 } }
I20260812 11:12:28.582605 6770 sys_catalog.cc:564] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:12:28.583899 6777 sys_catalog.cc:455] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b11244d9a967487ba96bdcf19d8b68eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b11244d9a967487ba96bdcf19d8b68eb" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36829 } } }
I20260812 11:12:28.584060 6777 sys_catalog.cc:458] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [sys.catalog]: This master's current role is: LEADER
I20260812 11:12:28.584455 6778 sys_catalog.cc:455] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader b11244d9a967487ba96bdcf19d8b68eb. Latest consensus state: current_term: 1 leader_uuid: "b11244d9a967487ba96bdcf19d8b68eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b11244d9a967487ba96bdcf19d8b68eb" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36829 } } }
I20260812 11:12:28.584558 6789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:12:28.584674 6778 sys_catalog.cc:458] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [sys.catalog]: This master's current role is: LEADER
I20260812 11:12:28.584810 6773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:28.569931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48058 (local address 127.17.44.254:36829)
0812 11:12:28.570518 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:28.570535 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:28.570650 (+ 115us) server_negotiation.cc:407] Connection header received
0812 11:12:28.571852 (+ 1202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:28.571869 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:28.572174 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:28.572513 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:28.573379 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:28.574368 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:28.575296 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:28.575824 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:28.575999 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:28.578824 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:28.578829 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:28.578842 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:28.578869 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:28.581016 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:28.581579 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:28.581585 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:28.581591 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:28.581726 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:28.582062 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:28.582066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:28.582067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:28.582506 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:28.582730 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:28.583098 (+ 368us) server_negotiation.cc:299] Negotiation successful
0812 11:12:28.583446 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":159,"threads_started":1}
I20260812 11:12:28.588071 6789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:12:28.594712 6789 catalog_manager.cc:1347] Generated new cluster ID: 9d7a0d22bad647f381a1ec834f08f692
I20260812 11:12:28.594815 6789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:12:28.605805 6789 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:12:28.606742 6789 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:12:28.613530 6789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb: Generated new TSK 0
I20260812 11:12:28.614413 6789 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
I20260812 11:12:28.653841 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:36829
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:46029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:12:28.777232 6808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:28.777490 6808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:28.777532 6808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:28.781605 6808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:28.781661 6808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:28.781742 6808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:12:28.789021 6808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:46029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:36829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:28.790207 6808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:28.791461 6808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:28.803331 6821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:28.803325 6815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:28.803323 6823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:28.804052 6808 server_base.cc:1034] running on GCE node
I20260812 11:12:28.804503 6808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:28.805137 6808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:28.806320 6808 hybrid_clock.cc:648] HybridClock initialized: now 1786533148806300 us; error 34 us; skew 500 ppm
Aug 12 11:12:28 dist-test-slave-drl0 krb5kdc[6636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533148, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:28.809423 6808 init.cc:377] Logged in from keytab as kudu/127.17.44.193@KRBTEST.COM (short username kudu)
I20260812 11:12:28.810237 6808 webserver.cc:460] Webserver started at http://127.17.44.193:46773/ using document root <none> and password file <none>
I20260812 11:12:28.810783 6808 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:28.810846 6808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:12:28.811102 6808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:12:28.813740 6808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "e9bcc97a954b4d69888b38749ee663c9"
format_stamp: "Formatted at 2026-08-12 11:12:28 on dist-test-slave-drl0"
server_key: "5106f05786ec90df4275b3ed6dd1620f"
server_key_iv: "64ad37224cdca5a9226a5e153b457684"
server_key_version: "encryptionkey@0"
I20260812 11:12:28.814586 6808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "e9bcc97a954b4d69888b38749ee663c9"
format_stamp: "Formatted at 2026-08-12 11:12:28 on dist-test-slave-drl0"
server_key: "5106f05786ec90df4275b3ed6dd1620f"
server_key_iv: "64ad37224cdca5a9226a5e153b457684"
server_key_version: "encryptionkey@0"
I20260812 11:12:28.818542 6808 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260812 11:12:28.821224 6832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:28.822641 6808 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:12:28.822752 6808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "e9bcc97a954b4d69888b38749ee663c9"
format_stamp: "Formatted at 2026-08-12 11:12:28 on dist-test-slave-drl0"
server_key: "5106f05786ec90df4275b3ed6dd1620f"
server_key_iv: "64ad37224cdca5a9226a5e153b457684"
server_key_version: "encryptionkey@0"
I20260812 11:12:28.822887 6808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:28.832122 6808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:28.835342 6808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:28.835547 6808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:28.836138 6808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:12:28.836949 6808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:12:28.836997 6808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:28.837034 6808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:12:28.837071 6808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:28.847469 6808 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:45743
I20260812 11:12:28.847564 7004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:45743 every 8 connection(s)
I20260812 11:12:28.848423 6808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:12:28.852501 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 6808
I20260812 11:12:28.852656 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:12:28.852977 17587 external_mini_cluster.cc:1428] Setting key 7b2cda7dacc6baf5685f99c747fb4825
Aug 12 11:12:28 dist-test-slave-drl0 krb5kdc[6636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533148, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.44.194@KRBTEST.COM; defaulting to no policy
I20260812 11:12:28.863736 6773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:28.850298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:38335 (local address 127.17.44.254:36829)
0812 11:12:28.850478 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:28.850483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:28.851411 (+ 928us) server_negotiation.cc:407] Connection header received
0812 11:12:28.852875 (+ 1464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:28.852880 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:28.852946 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:28.853040 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:28.854295 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:28.854992 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:28.856714 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:28.857010 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:28.857157 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:28.860020 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:28.860024 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:28.860026 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:28.860054 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:28.861535 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:28.862089 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:28.862099 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:28.862101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:28.862175 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:28.862524 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:28.862528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:28.862530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:28.862726 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:28.862839 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:28.863430 (+ 591us) server_negotiation.cc:299] Negotiation successful
0812 11:12:28.863565 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 11:12:28.864571 7008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:28.850648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.193:38335)
0812 11:12:28.851240 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:28.851281 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:28.852613 (+ 1332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:28.853221 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:28.853227 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:28.853654 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:28.854154 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:28.854167 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:28.855339 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:28.855346 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:28.856524 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:28.856542 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:28.857229 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:28.857234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:28.857541 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:28.858305 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:28.858323 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:28.859822 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:28.861676 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:28.861682 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:28.861698 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:28.861971 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:28.862274 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:28.862277 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:28.862279 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:28.862423 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:28.862851 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:28.862859 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:28.863190 (+ 331us) client_negotiation.cc:769] Sending connection context
0812 11:12:28.863388 (+ 198us) client_negotiation.cc:240] Negotiation successful
0812 11:12:28.863723 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":153,"threads_started":1}
I20260812 11:12:28.866386 7005 heartbeater.cc:347] Connected to a master server at 127.17.44.254:36829
I20260812 11:12:28.866659 7005 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.194@KRBTEST.COM" created.
I20260812 11:12:28.867242 7005 heartbeater.cc:511] Master 127.17.44.254:36829 requested a full tablet report, sending...
I20260812 11:12:28.869117 6710 ts_manager.cc:194] Registered new tserver with Master: e9bcc97a954b4d69888b38749ee663c9 (127.17.44.193:45743)
I20260812 11:12:28.870620 6710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.193@KRBTEST.COM'} at 127.17.44.193:38335
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
I20260812 11:12:28.886502 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:36829
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:46029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:12:29.013936 7013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:29.014183 7013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:29.014231 7013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:29.018787 7013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:29.018867 7013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:29.019006 7013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:12:29.024969 7013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:46029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:36829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:29.026023 7013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:29.027164 7013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:29.037914 7018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:29.037909 7024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:29.037909 7019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:29.038467 7013 server_base.cc:1034] running on GCE node
I20260812 11:12:29.038900 7013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:29.039657 7013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:29.040879 7013 hybrid_clock.cc:648] HybridClock initialized: now 1786533149040812 us; error 47 us; skew 500 ppm
Aug 12 11:12:29 dist-test-slave-drl0 krb5kdc[6636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533149, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:29.044593 7013 init.cc:377] Logged in from keytab as kudu/127.17.44.194@KRBTEST.COM (short username kudu)
I20260812 11:12:29.045902 7013 webserver.cc:460] Webserver started at http://127.17.44.194:37751/ using document root <none> and password file <none>
I20260812 11:12:29.046583 7013 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:29.046664 7013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:12:29.046845 7013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:12:29.049016 7013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "f8b256eb81ee4ceb93c293d70db13348"
format_stamp: "Formatted at 2026-08-12 11:12:29 on dist-test-slave-drl0"
server_key: "32b126ae723620c2280d54cacf8efdc5"
server_key_iv: "f6167c2544a70885f2ce1da869d654cd"
server_key_version: "encryptionkey@0"
I20260812 11:12:29.049548 7013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "f8b256eb81ee4ceb93c293d70db13348"
format_stamp: "Formatted at 2026-08-12 11:12:29 on dist-test-slave-drl0"
server_key: "32b126ae723620c2280d54cacf8efdc5"
server_key_iv: "f6167c2544a70885f2ce1da869d654cd"
server_key_version: "encryptionkey@0"
I20260812 11:12:29.053359 7013 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 11:12:29.056192 7034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:29.057644 7013 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 11:12:29.057750 7013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "f8b256eb81ee4ceb93c293d70db13348"
format_stamp: "Formatted at 2026-08-12 11:12:29 on dist-test-slave-drl0"
server_key: "32b126ae723620c2280d54cacf8efdc5"
server_key_iv: "f6167c2544a70885f2ce1da869d654cd"
server_key_version: "encryptionkey@0"
I20260812 11:12:29.057894 7013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:29.080796 7013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:29.085530 7013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:29.085868 7013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:29.086580 7013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:12:29.087604 7013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:12:29.087662 7013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:29.087708 7013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:12:29.087769 7013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:29.097734 7013 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:35327
I20260812 11:12:29.097756 7215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:35327 every 8 connection(s)
I20260812 11:12:29.098632 7013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:12:29.104609 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 7013
I20260812 11:12:29.104828 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:12:29.105185 17587 external_mini_cluster.cc:1428] Setting key 189b0c84581c0ae802277ee0e5a4d7ef
Aug 12 11:12:29 dist-test-slave-drl0 krb5kdc[6636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533149, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:12:29.114135 6773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:29.101160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:57435 (local address 127.17.44.254:36829)
0812 11:12:29.101327 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:29.101332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:29.102293 (+ 961us) server_negotiation.cc:407] Connection header received
0812 11:12:29.103845 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:29.103848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:29.103896 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:29.103977 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:29.105733 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.106414 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.107654 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.107876 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.107989 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:29.110520 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:29.110529 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:29.110531 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:29.110554 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:29.112009 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:29.112550 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:29.112558 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:29.112560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:29.112636 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:29.113001 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:29.113006 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:29.113008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:29.113203 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:29.113315 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:29.113829 (+ 514us) server_negotiation.cc:299] Negotiation successful
0812 11:12:29.113960 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 11:12:29.115144 7220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:29.101604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.194:57435)
0812 11:12:29.102123 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:29.102165 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:29.103612 (+ 1447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:29.104185 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:29.104192 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:29.104810 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:29.105542 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:29.105565 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.106573 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.106576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:29.107526 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:29.107537 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.108026 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.108031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:29.108196 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:29.108806 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:29.108822 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:29.110347 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:29.112135 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:29.112141 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:29.112156 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:29.112425 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:29.112754 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:29.112758 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:29.112760 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:29.112902 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:29.113316 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:29.113324 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:29.113564 (+ 240us) client_negotiation.cc:769] Sending connection context
0812 11:12:29.113762 (+ 198us) client_negotiation.cc:240] Negotiation successful
0812 11:12:29.114087 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":153,"threads_started":1}
WARNING: no policy specified for kudu/127.17.44.195@KRBTEST.COM; defaulting to no policy
I20260812 11:12:29.116917 7217 heartbeater.cc:347] Connected to a master server at 127.17.44.254:36829
I20260812 11:12:29.117202 7217 heartbeater.cc:464] Registering TS with master...
I20260812 11:12:29.117703 7217 heartbeater.cc:511] Master 127.17.44.254:36829 requested a full tablet report, sending...
I20260812 11:12:29.119181 6710 ts_manager.cc:194] Registered new tserver with Master: f8b256eb81ee4ceb93c293d70db13348 (127.17.44.194:35327)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.195@KRBTEST.COM" created.
I20260812 11:12:29.119998 6710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.194@KRBTEST.COM'} at 127.17.44.194:57435
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
I20260812 11:12:29.138806 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:36829
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:46029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:12:29.258858 7224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:29.259128 7224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:29.259171 7224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:29.263177 7224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:29.263231 7224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:29.263289 7224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:12:29.268870 7224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:46029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:36829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:29.269824 7224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:29.270778 7224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:29.281122 7233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:29.281140 7230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:29.281140 7229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:29.281801 7224 server_base.cc:1034] running on GCE node
I20260812 11:12:29.282253 7224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:29.283087 7224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:29.286484 7224 hybrid_clock.cc:648] HybridClock initialized: now 1786533149286421 us; error 73 us; skew 500 ppm
Aug 12 11:12:29 dist-test-slave-drl0 krb5kdc[6636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533149, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:29.289736 7224 init.cc:377] Logged in from keytab as kudu/127.17.44.195@KRBTEST.COM (short username kudu)
I20260812 11:12:29.290903 7224 webserver.cc:460] Webserver started at http://127.17.44.195:35077/ using document root <none> and password file <none>
I20260812 11:12:29.291651 7224 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:29.291743 7224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:12:29.291967 7224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:12:29.294171 7224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "1ac60ba6fecc43bc856fae0dc226066e"
format_stamp: "Formatted at 2026-08-12 11:12:29 on dist-test-slave-drl0"
server_key: "c4a03ead7f68e36e0958aa7204871ee6"
server_key_iv: "4df71ef46b902fb0e8a22eb43621a49c"
server_key_version: "encryptionkey@0"
I20260812 11:12:29.294749 7224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "1ac60ba6fecc43bc856fae0dc226066e"
format_stamp: "Formatted at 2026-08-12 11:12:29 on dist-test-slave-drl0"
server_key: "c4a03ead7f68e36e0958aa7204871ee6"
server_key_iv: "4df71ef46b902fb0e8a22eb43621a49c"
server_key_version: "encryptionkey@0"
I20260812 11:12:29.298866 7224 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260812 11:12:29.301705 7249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:29.303371 7224 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 11:12:29.303517 7224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "1ac60ba6fecc43bc856fae0dc226066e"
format_stamp: "Formatted at 2026-08-12 11:12:29 on dist-test-slave-drl0"
server_key: "c4a03ead7f68e36e0958aa7204871ee6"
server_key_iv: "4df71ef46b902fb0e8a22eb43621a49c"
server_key_version: "encryptionkey@0"
I20260812 11:12:29.303632 7224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:29.326972 7224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:29.330204 7224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:29.330371 7224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:29.331058 7224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:12:29.331893 7224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:12:29.331957 7224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:29.331996 7224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:12:29.332012 7224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:29.341979 7224 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:46013
I20260812 11:12:29.342058 7423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:46013 every 8 connection(s)
I20260812 11:12:29.342905 7224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:12:29.347040 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 7224
I20260812 11:12:29.347198 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:12:29.347589 17587 external_mini_cluster.cc:1428] Setting key ee8a14875542c944237280582ead34cc
Aug 12 11:12:29 dist-test-slave-drl0 krb5kdc[6636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533149, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:12:29.360617 6773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:29.345589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:58467 (local address 127.17.44.254:36829)
0812 11:12:29.345770 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:29.345784 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:29.346578 (+ 794us) server_negotiation.cc:407] Connection header received
0812 11:12:29.348387 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:29.348396 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:29.348442 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:29.348498 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:29.349963 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.350511 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.352366 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.352669 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.352774 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:29.356195 (+ 3421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:29.356200 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:29.356202 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:29.356246 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:29.358203 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:29.358886 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:29.358889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:29.358891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:29.359034 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:29.359409 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:29.359412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:29.359413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:29.359593 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:29.359672 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:29.360321 (+ 649us) server_negotiation.cc:299] Negotiation successful
0812 11:12:29.360430 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 11:12:29.361747 7429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:29.345927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.195:58467)
0812 11:12:29.346414 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:29.346456 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:29.348178 (+ 1722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:29.348672 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:29.348678 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:29.349123 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:29.349795 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:29.349816 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.350755 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.350761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:29.352141 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:29.352169 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.352896 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.352900 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:29.353177 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:29.353967 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:29.353984 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:29.355956 (+ 1972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:29.358344 (+ 2388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:29.358349 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:29.358361 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:29.358650 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:29.359134 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:29.359136 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:29.359138 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:29.359280 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:29.359768 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:29.359781 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:29.360023 (+ 242us) client_negotiation.cc:769] Sending connection context
0812 11:12:29.360185 (+ 162us) client_negotiation.cc:240] Negotiation successful
0812 11:12:29.360648 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":124,"threads_started":1}
I20260812 11:12:29.363360 7424 heartbeater.cc:347] Connected to a master server at 127.17.44.254:36829
I20260812 11:12:29.363596 7424 heartbeater.cc:464] Registering TS with master...
I20260812 11:12:29.364392 7424 heartbeater.cc:511] Master 127.17.44.254:36829 requested a full tablet report, sending...
I20260812 11:12:29.366420 6710 ts_manager.cc:194] Registered new tserver with Master: 1ac60ba6fecc43bc856fae0dc226066e (127.17.44.195:46013)
I20260812 11:12:29.367416 6710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.195@KRBTEST.COM'} at 127.17.44.195:58467
I20260812 11:12:29.374877 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:12:29.387511 6773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:29.377648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48060 (local address 127.17.44.254:36829)
0812 11:12:29.377915 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:29.377920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:29.377946 (+ 26us) server_negotiation.cc:407] Connection header received
0812 11:12:29.378004 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:29.378007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:29.378063 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:29.378250 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:29.379380 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.379902 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.381044 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.381264 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.381356 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:29.383008 (+ 1652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:29.383012 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:29.383022 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:29.383046 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:29.384963 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:29.385407 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:29.385410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:29.385411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:29.385493 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:29.385866 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:29.385869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:29.385870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:29.386095 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:29.386220 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:29.386607 (+ 387us) server_negotiation.cc:299] Negotiation successful
0812 11:12:29.387333 (+ 726us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260812 11:12:29.391134 6710 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:12:29.394296 6710 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:12:29.410885 7442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:29.403991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:46013 (local address 127.0.0.1:38958)
0812 11:12:29.405210 (+ 1219us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:29.405259 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:29.405394 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:29.406146 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:29.406152 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:29.406174 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:12:29.406349 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:29.406361 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.407807 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.407811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:29.409260 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:29.409271 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.410298 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.410302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:29.410421 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:29.410458 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 11:12:29.410595 (+ 137us) client_negotiation.cc:240] Negotiation successful
0812 11:12:29.410722 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1043,"mutex_wait_us":35,"thread_start_us":74,"threads_started":1}
I20260812 11:12:29.410892 7445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:29.404208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38958 (local address 127.17.44.195:46013)
0812 11:12:29.404965 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:29.404978 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:29.405759 (+ 781us) server_negotiation.cc:407] Connection header received
0812 11:12:29.405836 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:29.405841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:29.405994 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:29.406154 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:12:29.406479 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.407654 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.409414 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.410170 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.410304 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:29.410458 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:29.410639 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 11:12:29.410734 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":143,"threads_started":1}
I20260812 11:12:29.411504 7443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:29.404224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34580 (local address 127.17.44.193:45743)
0812 11:12:29.406000 (+ 1776us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:29.406007 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:29.406030 (+ 23us) server_negotiation.cc:407] Connection header received
0812 11:12:29.406110 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:29.406115 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:29.406258 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:29.406387 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:12:29.406967 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.408016 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.409486 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.410257 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.410380 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:29.410520 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:29.411235 (+ 715us) server_negotiation.cc:299] Negotiation successful
0812 11:12:29.411345 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1573,"thread_start_us":139,"threads_started":1}
I20260812 11:12:29.411605 7447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:29.403999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:45743 (local address 127.0.0.1:34580)
0812 11:12:29.405409 (+ 1410us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:29.405422 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:29.405497 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:29.406547 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:29.406556 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:29.406573 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:12:29.406786 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:29.406807 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.408146 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.408150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:29.409359 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:29.409367 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.410788 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.410794 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:29.411030 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:29.411077 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 11:12:29.411202 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 11:12:29.411316 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1354,"mutex_wait_us":525,"thread_start_us":91,"threads_started":1}
I20260812 11:12:29.412438 7446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:29.404621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:35327 (local address 127.0.0.1:40524)
0812 11:12:29.405611 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:29.405624 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:29.405689 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:29.406393 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:29.406398 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:29.406424 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:12:29.406524 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:29.406531 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.408070 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.408073 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:29.409185 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:29.409200 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.411996 (+ 2796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.412000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:29.412111 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:29.412137 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 11:12:29.412216 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 11:12:29.412296 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":936,"mutex_wait_us":96,"thread_start_us":284,"threads_started":1}
I20260812 11:12:29.412895 7449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:29.405447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40524 (local address 127.17.44.194:35327)
0812 11:12:29.405781 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:29.405795 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:29.405823 (+ 28us) server_negotiation.cc:407] Connection header received
0812 11:12:29.406022 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:29.406028 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:29.406224 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:29.406397 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:12:29.406640 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.407909 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.409332 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.410256 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.410380 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:29.410520 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:29.412633 (+ 2113us) server_negotiation.cc:299] Negotiation successful
0812 11:12:29.412733 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":126,"threads_started":1}
I20260812 11:12:29.414328 7318 tablet_service.cc:1467] Processing CreateTablet for tablet 99ea9726adca47a4b4ddae4725d2b895 (DEFAULT_TABLE table=test-table [id=f5c59f6c74154c92b4eb699f768c45e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:12:29.414384 6902 tablet_service.cc:1467] Processing CreateTablet for tablet 99ea9726adca47a4b4ddae4725d2b895 (DEFAULT_TABLE table=test-table [id=f5c59f6c74154c92b4eb699f768c45e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:12:29.415620 6902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99ea9726adca47a4b4ddae4725d2b895. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:12:29.415771 7113 tablet_service.cc:1467] Processing CreateTablet for tablet 99ea9726adca47a4b4ddae4725d2b895 (DEFAULT_TABLE table=test-table [id=f5c59f6c74154c92b4eb699f768c45e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:12:29.415812 7318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99ea9726adca47a4b4ddae4725d2b895. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:12:29.416692 7113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99ea9726adca47a4b4ddae4725d2b895. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:12:29.423311 7450 tablet_bootstrap.cc:492] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9: Bootstrap starting.
I20260812 11:12:29.424332 7456 tablet_bootstrap.cc:492] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e: Bootstrap starting.
I20260812 11:12:29.426663 7450 tablet_bootstrap.cc:654] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9: Neither blocks nor log segments found. Creating new log.
I20260812 11:12:29.427167 7456 tablet_bootstrap.cc:654] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e: Neither blocks nor log segments found. Creating new log.
I20260812 11:12:29.427464 7455 tablet_bootstrap.cc:492] T 99ea9726adca47a4b4ddae4725d2b895 P f8b256eb81ee4ceb93c293d70db13348: Bootstrap starting.
I20260812 11:12:29.427932 7456 log.cc:824] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e: Log is configured to *not* fsync() on all Append() calls
I20260812 11:12:29.427932 7450 log.cc:824] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9: Log is configured to *not* fsync() on all Append() calls
I20260812 11:12:29.429805 7455 tablet_bootstrap.cc:654] T 99ea9726adca47a4b4ddae4725d2b895 P f8b256eb81ee4ceb93c293d70db13348: Neither blocks nor log segments found. Creating new log.
I20260812 11:12:29.430074 7456 tablet_bootstrap.cc:492] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e: No bootstrap required, opened a new log
I20260812 11:12:29.430074 7450 tablet_bootstrap.cc:492] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9: No bootstrap required, opened a new log
I20260812 11:12:29.430270 7450 ts_tablet_manager.cc:1397] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 11:12:29.430270 7456 ts_tablet_manager.cc:1397] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 11:12:29.430612 7455 log.cc:824] T 99ea9726adca47a4b4ddae4725d2b895 P f8b256eb81ee4ceb93c293d70db13348: Log is configured to *not* fsync() on all Append() calls
I20260812 11:12:29.432220 7455 tablet_bootstrap.cc:492] T 99ea9726adca47a4b4ddae4725d2b895 P f8b256eb81ee4ceb93c293d70db13348: No bootstrap required, opened a new log
I20260812 11:12:29.432507 7455 ts_tablet_manager.cc:1397] T 99ea9726adca47a4b4ddae4725d2b895 P f8b256eb81ee4ceb93c293d70db13348: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260812 11:12:29.434417 7456 raft_consensus.cc:348] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } }
I20260812 11:12:29.434697 7456 raft_consensus.cc:374] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:12:29.434747 7456 raft_consensus.cc:729] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ac60ba6fecc43bc856fae0dc226066e, State: Initialized, Role: FOLLOWER
I20260812 11:12:29.434652 7450 raft_consensus.cc:348] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } }
I20260812 11:12:29.434883 7450 raft_consensus.cc:374] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:12:29.434971 7450 raft_consensus.cc:729] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bcc97a954b4d69888b38749ee663c9, State: Initialized, Role: FOLLOWER
I20260812 11:12:29.435286 7456 consensus_queue.cc:260] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } }
I20260812 11:12:29.435362 7450 consensus_queue.cc:260] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } }
I20260812 11:12:29.436307 7456 ts_tablet_manager.cc:1428] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 11:12:29.436317 7450 ts_tablet_manager.cc:1428] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 11:12:29.436709 7424 heartbeater.cc:503] Master 127.17.44.254:36829 was elected leader, sending a full tablet report...
I20260812 11:12:29.436866 7005 heartbeater.cc:503] Master 127.17.44.254:36829 was elected leader, sending a full tablet report...
I20260812 11:12:29.437261 7455 raft_consensus.cc:348] T 99ea9726adca47a4b4ddae4725d2b895 P f8b256eb81ee4ceb93c293d70db13348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } }
I20260812 11:12:29.437520 7455 raft_consensus.cc:374] T 99ea9726adca47a4b4ddae4725d2b895 P f8b256eb81ee4ceb93c293d70db13348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:12:29.437578 7455 raft_consensus.cc:729] T 99ea9726adca47a4b4ddae4725d2b895 P f8b256eb81ee4ceb93c293d70db13348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8b256eb81ee4ceb93c293d70db13348, State: Initialized, Role: FOLLOWER
I20260812 11:12:29.437979 7455 consensus_queue.cc:260] T 99ea9726adca47a4b4ddae4725d2b895 P f8b256eb81ee4ceb93c293d70db13348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } }
I20260812 11:12:29.438606 7217 heartbeater.cc:503] Master 127.17.44.254:36829 was elected leader, sending a full tablet report...
I20260812 11:12:29.438949 7455 ts_tablet_manager.cc:1428] T 99ea9726adca47a4b4ddae4725d2b895 P f8b256eb81ee4ceb93c293d70db13348: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260812 11:12:29.468529 7463 raft_consensus.cc:482] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:12:29.468734 7463 raft_consensus.cc:504] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } }
I20260812 11:12:29.470295 7463 leader_election.cc:288] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9bcc97a954b4d69888b38749ee663c9 (127.17.44.193:45743), f8b256eb81ee4ceb93c293d70db13348 (127.17.44.194:35327)
I20260812 11:12:29.474382 7429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:29.470586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:35327 (local address 127.17.44.195:54957)
0812 11:12:29.470730 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:29.470748 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:29.470853 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:29.471241 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:29.471244 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:29.471267 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:12:29.471408 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:29.471416 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.472271 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.472275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:29.473176 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:29.473186 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.473927 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.473932 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:29.474041 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:29.474068 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 11:12:29.474167 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 11:12:29.474238 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":39}
I20260812 11:12:29.474462 7449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:29.470829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:54957 (local address 127.17.44.194:35327)
0812 11:12:29.470968 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:29.470973 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:29.470987 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:12:29.471042 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:29.471046 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:29.471094 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:29.471190 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:12:29.471547 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.472142 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.473319 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.473797 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.473899 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:29.473949 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:29.474246 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 11:12:29.474332 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 11:12:29.474895 7443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:29.470741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:37833 (local address 127.17.44.193:45743)
0812 11:12:29.470880 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:29.470884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:29.471219 (+ 335us) server_negotiation.cc:407] Connection header received
0812 11:12:29.471282 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:29.471284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:29.471328 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:29.471418 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:12:29.471786 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.472337 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.473486 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.474090 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.474207 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:29.474250 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:29.474556 (+ 306us) server_negotiation.cc:299] Negotiation successful
0812 11:12:29.474751 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 11:12:29.475381 7468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:29.470586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:45743 (local address 127.17.44.195:37833)
0812 11:12:29.471095 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:29.471108 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:29.471209 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:29.471497 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:29.471502 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:29.471525 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:12:29.471642 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:29.471649 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.472467 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.472470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:29.473321 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:29.473331 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.474257 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.474260 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:29.474373 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:29.474398 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 11:12:29.475129 (+ 731us) client_negotiation.cc:240] Negotiation successful
0812 11:12:29.475208 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":77,"thread_start_us":196,"threads_started":1}
I20260812 11:12:29.475323 7145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ea9726adca47a4b4ddae4725d2b895" candidate_uuid: "1ac60ba6fecc43bc856fae0dc226066e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8b256eb81ee4ceb93c293d70db13348" is_pre_election: true
I20260812 11:12:29.475659 7145 raft_consensus.cc:2454] T 99ea9726adca47a4b4ddae4725d2b895 P f8b256eb81ee4ceb93c293d70db13348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ac60ba6fecc43bc856fae0dc226066e in term 0.
I20260812 11:12:29.476151 6930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ea9726adca47a4b4ddae4725d2b895" candidate_uuid: "1ac60ba6fecc43bc856fae0dc226066e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bcc97a954b4d69888b38749ee663c9" is_pre_election: true
I20260812 11:12:29.476213 7253 leader_election.cc:302] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ac60ba6fecc43bc856fae0dc226066e, f8b256eb81ee4ceb93c293d70db13348; no voters:
I20260812 11:12:29.476423 6930 raft_consensus.cc:2454] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ac60ba6fecc43bc856fae0dc226066e in term 0.
I20260812 11:12:29.476517 7463 raft_consensus.cc:2781] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:12:29.476624 7463 raft_consensus.cc:482] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:12:29.476732 7463 raft_consensus.cc:3037] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:12:29.477890 7463 raft_consensus.cc:504] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } }
I20260812 11:12:29.478235 7463 leader_election.cc:288] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e [CANDIDATE]: Term 1 election: Requested vote from peers e9bcc97a954b4d69888b38749ee663c9 (127.17.44.193:45743), f8b256eb81ee4ceb93c293d70db13348 (127.17.44.194:35327)
I20260812 11:12:29.478613 6930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ea9726adca47a4b4ddae4725d2b895" candidate_uuid: "1ac60ba6fecc43bc856fae0dc226066e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9bcc97a954b4d69888b38749ee663c9"
I20260812 11:12:29.478775 6930 raft_consensus.cc:3037] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:12:29.479836 6930 raft_consensus.cc:2454] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ac60ba6fecc43bc856fae0dc226066e in term 1.
I20260812 11:12:29.480108 7145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ea9726adca47a4b4ddae4725d2b895" candidate_uuid: "1ac60ba6fecc43bc856fae0dc226066e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8b256eb81ee4ceb93c293d70db13348"
I20260812 11:12:29.480273 7145 raft_consensus.cc:3037] T 99ea9726adca47a4b4ddae4725d2b895 P f8b256eb81ee4ceb93c293d70db13348 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:12:29.480410 7252 leader_election.cc:302] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ac60ba6fecc43bc856fae0dc226066e, e9bcc97a954b4d69888b38749ee663c9; no voters:
I20260812 11:12:29.480625 7463 raft_consensus.cc:2781] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:12:29.480854 7463 raft_consensus.cc:686] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e [term 1 LEADER]: Becoming Leader. State: Replica: 1ac60ba6fecc43bc856fae0dc226066e, State: Running, Role: LEADER
I20260812 11:12:29.481196 7463 consensus_queue.cc:237] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } }
I20260812 11:12:29.481581 7145 raft_consensus.cc:2454] T 99ea9726adca47a4b4ddae4725d2b895 P f8b256eb81ee4ceb93c293d70db13348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ac60ba6fecc43bc856fae0dc226066e in term 1.
I20260812 11:12:29.484774 6708 catalog_manager.cc:5495] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ac60ba6fecc43bc856fae0dc226066e (127.17.44.195). New cstate: current_term: 1 leader_uuid: "1ac60ba6fecc43bc856fae0dc226066e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } health_report { overall_health: UNKNOWN } } }
I20260812 11:12:29.514361 7445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:29.509757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38966 (local address 127.17.44.195:46013)
0812 11:12:29.509890 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:29.509895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:29.509914 (+ 19us) server_negotiation.cc:407] Connection header received
0812 11:12:29.510095 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:29.510097 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:29.510154 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:29.510249 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:12:29.510505 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.511198 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.512554 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:29.512780 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:29.512872 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:29.513374 (+ 502us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:12:29.513804 (+ 430us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:12:29.513913 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:29.514078 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 11:12:29.514134 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 11:12:29.521251 7145 raft_consensus.cc:1261] T 99ea9726adca47a4b4ddae4725d2b895 P f8b256eb81ee4ceb93c293d70db13348 [term 1 FOLLOWER]: Refusing update from remote peer 1ac60ba6fecc43bc856fae0dc226066e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:12:29.521252 6930 raft_consensus.cc:1261] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Refusing update from remote peer 1ac60ba6fecc43bc856fae0dc226066e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:12:29.522037 7469 consensus_queue.cc:1035] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:12:29.522272 7463 consensus_queue.cc:1035] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:12:29.530695 7472 mvcc.cc:205] Tried to move back new op lower bound from 7317639780431372288 to 7317639780277809152. Current Snapshot: MvccSnapshot[applied={T|T < 7317639780431372288}]
I20260812 11:12:29.531931 7474 mvcc.cc:205] Tried to move back new op lower bound from 7317639780431372288 to 7317639780277809152. Current Snapshot: MvccSnapshot[applied={T|T < 7317639780431372288}]
I20260812 11:12:29.532482 7473 mvcc.cc:205] Tried to move back new op lower bound from 7317639780431372288 to 7317639780277809152. Current Snapshot: MvccSnapshot[applied={T|T < 7317639780431372288 or (T in {7317639780431372288})}]
I20260812 11:12:29.543150 6710 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:12:29.543351 6710 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:12:29.546473 6710 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb: Sending DeleteTablet for 3 replicas of tablet 99ea9726adca47a4b4ddae4725d2b895
I20260812 11:12:29.547662 7318 tablet_service.cc:1514] Processing DeleteTablet for tablet 99ea9726adca47a4b4ddae4725d2b895 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:12:29 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:38958
I20260812 11:12:29.547716 6902 tablet_service.cc:1514] Processing DeleteTablet for tablet 99ea9726adca47a4b4ddae4725d2b895 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:12:29 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:34580
I20260812 11:12:29.548221 7490 tablet_replica.cc:330] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9: stopping tablet replica
I20260812 11:12:29.548293 7113 tablet_service.cc:1514] Processing DeleteTablet for tablet 99ea9726adca47a4b4ddae4725d2b895 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:12:29 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:40524
I20260812 11:12:29.548593 7490 raft_consensus.cc:2229] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:12:29.548941 7491 tablet_replica.cc:330] T 99ea9726adca47a4b4ddae4725d2b895 P f8b256eb81ee4ceb93c293d70db13348: stopping tablet replica
I20260812 11:12:29.548974 7490 raft_consensus.cc:2258] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:12:29.549405 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 6808
I20260812 11:12:29.549419 7488 tablet_replica.cc:330] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e: stopping tablet replica
I20260812 11:12:29.549681 7488 raft_consensus.cc:2229] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:12:29.550081 7488 raft_consensus.cc:2258] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:12:29.555859 7491 raft_consensus.cc:2229] T 99ea9726adca47a4b4ddae4725d2b895 P f8b256eb81ee4ceb93c293d70db13348 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:12:29.556202 7491 raft_consensus.cc:2258] T 99ea9726adca47a4b4ddae4725d2b895 P f8b256eb81ee4ceb93c293d70db13348 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:12:29.557792 7491 ts_tablet_manager.cc:1905] T 99ea9726adca47a4b4ddae4725d2b895 P f8b256eb81ee4ceb93c293d70db13348: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:12:29.561700 7491 ts_tablet_manager.cc:1918] T 99ea9726adca47a4b4ddae4725d2b895 P f8b256eb81ee4ceb93c293d70db13348: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:12:29.561820 7491 log.cc:1196] T 99ea9726adca47a4b4ddae4725d2b895 P f8b256eb81ee4ceb93c293d70db13348: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/99ea9726adca47a4b4ddae4725d2b895
I20260812 11:12:29.562182 7491 ts_tablet_manager.cc:1939] T 99ea9726adca47a4b4ddae4725d2b895 P f8b256eb81ee4ceb93c293d70db13348: Deleting consensus metadata
I20260812 11:12:29.563755 6691 catalog_manager.cc:4841] TS f8b256eb81ee4ceb93c293d70db13348 (127.17.44.194:35327): tablet 99ea9726adca47a4b4ddae4725d2b895 (table test-table [id=f5c59f6c74154c92b4eb699f768c45e2]) successfully deleted
I20260812 11:12:29.564479 7488 ts_tablet_manager.cc:1905] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 11:12:29.565851 6687 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.44.193:45743 (error 108)
I20260812 11:12:29.567371 7447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:29.567093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:45743 (local address 127.0.0.1:34592)
0812 11:12:29.567224 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:29.567293 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.193:45743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 11:12:29.567370 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 7013
I20260812 11:12:29.567432 7488 ts_tablet_manager.cc:1918] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:12:29.567972 7488 log.cc:1196] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/99ea9726adca47a4b4ddae4725d2b895
I20260812 11:12:29.568374 7488 ts_tablet_manager.cc:1939] T 99ea9726adca47a4b4ddae4725d2b895 P 1ac60ba6fecc43bc856fae0dc226066e: Deleting consensus metadata
I20260812 11:12:29.569733 6684 catalog_manager.cc:4841] TS 1ac60ba6fecc43bc856fae0dc226066e (127.17.44.195:46013): tablet 99ea9726adca47a4b4ddae4725d2b895 (table test-table [id=f5c59f6c74154c92b4eb699f768c45e2]) successfully deleted
W20260812 11:12:29.572180 6687 catalog_manager.cc:4584] TS e9bcc97a954b4d69888b38749ee663c9 (127.17.44.193:45743): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 99ea9726adca47a4b4ddae4725d2b895: Network error: Client connection negotiation failed: client connection to 127.17.44.193:45743: connect: Connection refused (error 111)
I20260812 11:12:29.579034 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 7224
I20260812 11:12:29.593096 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 6655
I20260812 11:12:29.606650 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:36829
--webserver_interface=127.17.44.254
--webserver_port=43893
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:46029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:36829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:12:29.728446 7496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:29.728708 7496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:29.728751 7496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:29.733376 7496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:12:29.733438 7496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:29.733457 7496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:29.733474 7496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:12:29.733489 7496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:12:29.741214 7496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:46029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:36829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:36829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=43893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:29.742605 7496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:29.744297 7496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:29.759222 7511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:29.759236 7509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:29.759235 7513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:29.759338 7496 server_base.cc:1034] running on GCE node
I20260812 11:12:29.760099 7496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:29.761221 7496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:29.762673 7496 hybrid_clock.cc:648] HybridClock initialized: now 1786533149762591 us; error 82 us; skew 500 ppm
Aug 12 11:12:29 dist-test-slave-drl0 krb5kdc[6636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533149, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:29.766615 7496 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:12:29.767982 7496 webserver.cc:460] Webserver started at http://127.17.44.254:43893/ using document root <none> and password file <none>
I20260812 11:12:29.768791 7496 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:29.768884 7496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:12:29.773981 7496 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:12:29.776645 7521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:29.778015 7496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 11:12:29.778137 7496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "b11244d9a967487ba96bdcf19d8b68eb"
format_stamp: "Formatted at 2026-08-12 11:12:28 on dist-test-slave-drl0"
server_key: "94d5f5d03ebfa615a0832c5d8962236d"
server_key_iv: "a6015e6f897679ff3d48f6bbb7b0bac7"
server_key_version: "encryptionkey@0"
I20260812 11:12:29.778859 7496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:29.811213 7496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:29.815280 7496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:29.815609 7496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:29.824302 7496 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:36829
I20260812 11:12:29.824666 7606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:36829 every 8 connection(s)
I20260812 11:12:29.825870 7496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:12:29.830569 7607 sys_catalog.cc:263] Verifying existing consensus state
I20260812 11:12:29.832268 7607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb: Bootstrap starting.
I20260812 11:12:29.835064 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 7496
I20260812 11:12:29.835538 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:45743
--local_ip_for_outbound_sockets=127.17.44.193
--tserver_master_addrs=127.17.44.254:36829
--webserver_port=46773
--webserver_interface=127.17.44.193
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:46029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:12:29.854614 7607 log.cc:824] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb: Log is configured to *not* fsync() on all Append() calls
I20260812 11:12:29.865335 7607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 11:12:29.865736 7607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb: Bootstrap complete.
I20260812 11:12:29.870501 7607 raft_consensus.cc:348] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b11244d9a967487ba96bdcf19d8b68eb" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36829 } }
I20260812 11:12:29.870880 7607 raft_consensus.cc:729] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b11244d9a967487ba96bdcf19d8b68eb, State: Initialized, Role: FOLLOWER
I20260812 11:12:29.871440 7607 consensus_queue.cc:260] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b11244d9a967487ba96bdcf19d8b68eb" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36829 } }
I20260812 11:12:29.871594 7607 raft_consensus.cc:388] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:12:29.871649 7607 raft_consensus.cc:482] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:12:29.871739 7607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 1 FOLLOWER]: Advancing to term 2
I20260812 11:12:29.873648 7607 raft_consensus.cc:504] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b11244d9a967487ba96bdcf19d8b68eb" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36829 } }
I20260812 11:12:29.873991 7607 leader_election.cc:302] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b11244d9a967487ba96bdcf19d8b68eb; no voters:
I20260812 11:12:29.874336 7607 leader_election.cc:288] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 11:12:29.874413 7614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 2 FOLLOWER]: Leader election won for term 2
I20260812 11:12:29.874610 7614 raft_consensus.cc:686] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 2 LEADER]: Becoming Leader. State: Replica: b11244d9a967487ba96bdcf19d8b68eb, State: Running, Role: LEADER
I20260812 11:12:29.875030 7614 consensus_queue.cc:237] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b11244d9a967487ba96bdcf19d8b68eb" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36829 } }
I20260812 11:12:29.875790 7607 sys_catalog.cc:564] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:12:29.876458 7615 sys_catalog.cc:455] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b11244d9a967487ba96bdcf19d8b68eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b11244d9a967487ba96bdcf19d8b68eb" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36829 } } }
I20260812 11:12:29.876586 7615 sys_catalog.cc:458] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [sys.catalog]: This master's current role is: LEADER
I20260812 11:12:29.877053 7616 sys_catalog.cc:455] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader b11244d9a967487ba96bdcf19d8b68eb. Latest consensus state: current_term: 2 leader_uuid: "b11244d9a967487ba96bdcf19d8b68eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b11244d9a967487ba96bdcf19d8b68eb" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36829 } } }
I20260812 11:12:29.877200 7616 sys_catalog.cc:458] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [sys.catalog]: This master's current role is: LEADER
I20260812 11:12:29.877501 7624 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:12:29.882997 7624 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:12:29.883934 7624 catalog_manager.cc:1259] Loaded cluster ID: 9d7a0d22bad647f381a1ec834f08f692
I20260812 11:12:29.883997 7624 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:12:29.885572 7624 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 11:12:29.890614 7624 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 11:12:29.896497 7624 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:12:29.897292 7624 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb: Loaded TSK: 0
I20260812 11:12:29.898505 7624 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 11:12:29.974903 7612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:29.975256 7612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:29.975335 7612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:29.979800 7612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:29.979864 7612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:29.979923 7612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:12:29.985450 7612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:46029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:45743
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=46773
--tserver_master_addrs=127.17.44.254:36829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:29.986488 7612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:29.987722 7612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:29.998816 7655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:29.998821 7660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:29.998966 7654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:29.999435 7612 server_base.cc:1034] running on GCE node
I20260812 11:12:29.999893 7612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:30.000839 7612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:30.002331 7612 hybrid_clock.cc:648] HybridClock initialized: now 1786533150002275 us; error 53 us; skew 500 ppm
Aug 12 11:12:30 dist-test-slave-drl0 krb5kdc[6636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533150, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:30.006660 7612 init.cc:377] Logged in from keytab as kudu/127.17.44.193@KRBTEST.COM (short username kudu)
I20260812 11:12:30.008450 7612 webserver.cc:460] Webserver started at http://127.17.44.193:46773/ using document root <none> and password file <none>
I20260812 11:12:30.009334 7612 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:30.009441 7612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:12:30.014875 7612 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:12:30.017267 7672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:30.019011 7612 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:12:30.019148 7612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "e9bcc97a954b4d69888b38749ee663c9"
format_stamp: "Formatted at 2026-08-12 11:12:28 on dist-test-slave-drl0"
server_key: "5106f05786ec90df4275b3ed6dd1620f"
server_key_iv: "64ad37224cdca5a9226a5e153b457684"
server_key_version: "encryptionkey@0"
I20260812 11:12:30.019711 7612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:30.060286 7612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:30.063812 7612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:30.064031 7612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:30.064692 7612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:12:30.065893 7680 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 11:12:30.070271 7612 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 11:12:30.070379 7612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260812 11:12:30.070498 7612 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 11:12:30.072178 7612 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 11:12:30.072256 7612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260812 11:12:30.072659 7680 tablet_bootstrap.cc:492] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9: Bootstrap starting.
I20260812 11:12:30.083561 7612 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:45743
I20260812 11:12:30.083657 7857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:45743 every 8 connection(s)
I20260812 11:12:30.085173 7612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:12:30.085493 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 7612
I20260812 11:12:30.086055 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:35327
--local_ip_for_outbound_sockets=127.17.44.194
--tserver_master_addrs=127.17.44.254:36829
--webserver_port=37751
--webserver_interface=127.17.44.194
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:46029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:12:30.095480 7680 log.cc:824] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9: Log is configured to *not* fsync() on all Append() calls
Aug 12 11:12:30 dist-test-slave-drl0 krb5kdc[6636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533150, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:12:30.104364 7680 tablet_bootstrap.cc:492] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 11:12:30.104844 7680 tablet_bootstrap.cc:492] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9: Bootstrap complete.
I20260812 11:12:30.106232 7680 ts_tablet_manager.cc:1397] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9: Time spent bootstrapping tablet: real 0.034s user 0.026s sys 0.003s
I20260812 11:12:30.107324 7863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:30.088131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.193:38481)
0812 11:12:30.089318 (+ 1187us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:30.089364 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:30.090217 (+ 853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:30.091403 (+ 1186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:30.091411 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:30.092249 (+ 838us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:30.092928 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:30.092950 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.094683 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.094687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:30.096083 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:30.096104 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.097533 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.097538 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:30.097729 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:30.098534 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:30.098556 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:30.100757 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:30.103486 (+ 2729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:30.103492 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:30.103506 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:30.103846 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:30.104266 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:30.104269 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:30.104271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:30.104440 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:30.105260 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:30.105267 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:30.105654 (+ 387us) client_negotiation.cc:769] Sending connection context
0812 11:12:30.105869 (+ 215us) client_negotiation.cc:240] Negotiation successful
0812 11:12:30.106205 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"thread_start_us":203,"threads_started":1}
I20260812 11:12:30.107825 7861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:30.087881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:38481 (local address 127.17.44.254:36829)
0812 11:12:30.088799 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:30.088832 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:30.089515 (+ 683us) server_negotiation.cc:407] Connection header received
0812 11:12:30.090647 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:30.090657 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:30.091016 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:30.091339 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:30.093214 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.094455 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.096284 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.097318 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.097569 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:30.101031 (+ 3462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:30.101039 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:30.101055 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:30.101090 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:30.103345 (+ 2255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:30.103978 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:30.103985 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:30.103990 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:30.104156 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:30.104578 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:30.104582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:30.104583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:30.105091 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:30.105234 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:30.106006 (+ 772us) server_negotiation.cc:299] Negotiation successful
0812 11:12:30.106318 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":138,"threads_started":1}
I20260812 11:12:30.110146 7859 heartbeater.cc:347] Connected to a master server at 127.17.44.254:36829
I20260812 11:12:30.110376 7859 heartbeater.cc:464] Registering TS with master...
I20260812 11:12:30.110023 7680 raft_consensus.cc:348] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } }
I20260812 11:12:30.110559 7680 raft_consensus.cc:729] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bcc97a954b4d69888b38749ee663c9, State: Initialized, Role: FOLLOWER
I20260812 11:12:30.110973 7859 heartbeater.cc:511] Master 127.17.44.254:36829 requested a full tablet report, sending...
I20260812 11:12:30.111065 7680 consensus_queue.cc:260] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } }
I20260812 11:12:30.112133 7680 ts_tablet_manager.cc:1428] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260812 11:12:30.113677 7554 ts_manager.cc:194] Registered new tserver with Master: e9bcc97a954b4d69888b38749ee663c9 (127.17.44.193:45743)
I20260812 11:12:30.117260 7554 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.193@KRBTEST.COM'} at 127.17.44.193:38481
I20260812 11:12:30.119876 7859 heartbeater.cc:503] Master 127.17.44.254:36829 was elected leader, sending a full tablet report...
Aug 12 11:12:30 dist-test-slave-drl0 krb5kdc[6636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533149, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for kudu/127.17.44.193@KRBTEST.COM
I20260812 11:12:30.127709 7867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:30.116689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:45743 (local address 127.0.0.1:34604)
0812 11:12:30.117322 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:30.117365 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:30.117520 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:30.118054 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:30.118062 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:30.118485 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:30.118591 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:30.118597 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.119630 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.119634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:30.120397 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:30.120408 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.121000 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.121004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:30.121121 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:30.121707 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:30.121728 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:30.123754 (+ 2026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:30.125713 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:30.125722 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:30.125728 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:30.126031 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:30.126382 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:30.126385 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:30.126387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:30.126483 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:30.127073 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:30.127079 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:30.127201 (+ 122us) client_negotiation.cc:769] Sending connection context
0812 11:12:30.127353 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 11:12:30.127535 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":112,"threads_started":1}
I20260812 11:12:30.127737 7868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:30.116817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34604 (local address 127.17.44.193:45743)
0812 11:12:30.117504 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:30.117539 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:30.117564 (+ 25us) server_negotiation.cc:407] Connection header received
0812 11:12:30.117668 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:30.117673 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:30.117878 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:30.118005 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:30.118721 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.119510 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.120516 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.120842 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.120985 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:30.123934 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:30.123943 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:30.123953 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:30.123974 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:30.125556 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:30.126174 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:30.126180 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:30.126181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:30.126265 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:30.126585 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:30.126587 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:30.126588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:30.126871 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:30.127075 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:30.127475 (+ 400us) server_negotiation.cc:299] Negotiation successful
0812 11:12:30.127618 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":136,"threads_started":1}
I20260812 11:12:30.128887 7745 tablet_service.cc:1514] Processing DeleteTablet for tablet 99ea9726adca47a4b4ddae4725d2b895 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:12:29 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:34604
I20260812 11:12:30.129722 7870 tablet_replica.cc:330] stopping tablet replica
I20260812 11:12:30.130025 7870 raft_consensus.cc:2229] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:12:30.130288 7870 raft_consensus.cc:2258] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:12:30.131340 7870 ts_tablet_manager.cc:1905] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:12:30.135525 7870 ts_tablet_manager.cc:1918] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:12:30.135664 7870 log.cc:1196] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/wal/wals/99ea9726adca47a4b4ddae4725d2b895
I20260812 11:12:30.135974 7870 ts_tablet_manager.cc:1939] T 99ea9726adca47a4b4ddae4725d2b895 P e9bcc97a954b4d69888b38749ee663c9: Deleting consensus metadata
I20260812 11:12:30.137784 7525 catalog_manager.cc:4841] TS e9bcc97a954b4d69888b38749ee663c9 (127.17.44.193:45743): tablet 99ea9726adca47a4b4ddae4725d2b895 (table test-table [id=f5c59f6c74154c92b4eb699f768c45e2]) successfully deleted
W20260812 11:12:30.221108 7862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:30.221609 7862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:30.221715 7862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:30.227415 7862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:30.227527 7862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:30.227638 7862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:12:30.233340 7862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:46029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:35327
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=37751
--tserver_master_addrs=127.17.44.254:36829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:30.235086 7862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:30.236639 7862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 11:12:30.247622 7862 server_base.cc:1034] running on GCE node
W20260812 11:12:30.247562 7877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:30.247610 7878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:30.247562 7881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:30.248677 7862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:30.249641 7862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:30.250859 7862 hybrid_clock.cc:648] HybridClock initialized: now 1786533150250837 us; error 35 us; skew 500 ppm
Aug 12 11:12:30 dist-test-slave-drl0 krb5kdc[6636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533150, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:30.254688 7862 init.cc:377] Logged in from keytab as kudu/127.17.44.194@KRBTEST.COM (short username kudu)
I20260812 11:12:30.255947 7862 webserver.cc:460] Webserver started at http://127.17.44.194:37751/ using document root <none> and password file <none>
I20260812 11:12:30.256669 7862 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:30.256789 7862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:12:30.261237 7862 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:12:30.264273 7894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:30.265837 7862 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:12:30.266000 7862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "f8b256eb81ee4ceb93c293d70db13348"
format_stamp: "Formatted at 2026-08-12 11:12:29 on dist-test-slave-drl0"
server_key: "32b126ae723620c2280d54cacf8efdc5"
server_key_iv: "f6167c2544a70885f2ce1da869d654cd"
server_key_version: "encryptionkey@0"
I20260812 11:12:30.266577 7862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:30.280196 7862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:30.284683 7862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:30.284893 7862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:30.285476 7862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:12:30.286391 7862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:12:30.286455 7862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:30.286496 7862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:12:30.286523 7862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:30.297335 7862 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:35327
I20260812 11:12:30.297370 8063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:35327 every 8 connection(s)
I20260812 11:12:30.298362 7862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:12:30.307533 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 7862
I20260812 11:12:30.308094 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:46013
--local_ip_for_outbound_sockets=127.17.44.195
--tserver_master_addrs=127.17.44.254:36829
--webserver_port=35077
--webserver_interface=127.17.44.195
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:46029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 11:12:30 dist-test-slave-drl0 krb5kdc[6636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533150, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:12:30.315742 7861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:30.300740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:37665 (local address 127.17.44.254:36829)
0812 11:12:30.300929 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:30.300934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:30.302168 (+ 1234us) server_negotiation.cc:407] Connection header received
0812 11:12:30.303730 (+ 1562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:30.303733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:30.303781 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:30.303861 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:30.305626 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.306279 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.307685 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.308004 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.308119 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:30.310799 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:30.310806 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:30.310810 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:30.310844 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:30.312641 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:30.313254 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:30.313261 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:30.313266 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:30.313377 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:30.313817 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:30.313824 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:30.313828 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:30.314221 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:30.314443 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:30.315322 (+ 879us) server_negotiation.cc:299] Negotiation successful
0812 11:12:30.315486 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 11:12:30.316358 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:30.301269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.194:37665)
0812 11:12:30.301880 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:30.301941 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:30.303483 (+ 1542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:30.304024 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:30.304034 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:30.304572 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:30.305337 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:30.305360 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.306564 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.306568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:30.307528 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:30.307538 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.308160 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.308164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:30.308380 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:30.308996 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:30.309012 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:30.310627 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:30.312771 (+ 2144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:30.312775 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:30.312787 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:30.313132 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:30.313569 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:30.313571 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:30.313573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:30.313721 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:30.314746 (+ 1025us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:30.314751 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:30.315062 (+ 311us) client_negotiation.cc:769] Sending connection context
0812 11:12:30.315231 (+ 169us) client_negotiation.cc:240] Negotiation successful
0812 11:12:30.315494 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":128,"threads_started":1}
I20260812 11:12:30.317629 8064 heartbeater.cc:347] Connected to a master server at 127.17.44.254:36829
I20260812 11:12:30.317863 8064 heartbeater.cc:464] Registering TS with master...
I20260812 11:12:30.318459 8064 heartbeater.cc:511] Master 127.17.44.254:36829 requested a full tablet report, sending...
I20260812 11:12:30.319612 7554 ts_manager.cc:194] Registered new tserver with Master: f8b256eb81ee4ceb93c293d70db13348 (127.17.44.194:35327)
I20260812 11:12:30.320211 7554 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.194@KRBTEST.COM'} at 127.17.44.194:37665
W20260812 11:12:30.433094 8067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:30.433357 8067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:30.433398 8067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:30.437680 8067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:30.437757 8067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:30.437817 8067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:12:30.444008 8067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:46029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:46013
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=35077
--tserver_master_addrs=127.17.44.254:36829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:30.445523 8067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:30.447036 8067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:30.457340 8075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:30.457357 8076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:30.457540 8080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:30.457887 8067 server_base.cc:1034] running on GCE node
I20260812 11:12:30.458293 8067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:30.459038 8067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:30.460251 8067 hybrid_clock.cc:648] HybridClock initialized: now 1786533150460233 us; error 35 us; skew 500 ppm
Aug 12 11:12:30 dist-test-slave-drl0 krb5kdc[6636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533150, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:30.463893 8067 init.cc:377] Logged in from keytab as kudu/127.17.44.195@KRBTEST.COM (short username kudu)
I20260812 11:12:30.465126 8067 webserver.cc:460] Webserver started at http://127.17.44.195:35077/ using document root <none> and password file <none>
I20260812 11:12:30.465744 8067 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:30.465865 8067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:12:30.470093 8067 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:12:30.472368 8088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:30.473770 8067 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 11:12:30.473932 8067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "1ac60ba6fecc43bc856fae0dc226066e"
format_stamp: "Formatted at 2026-08-12 11:12:29 on dist-test-slave-drl0"
server_key: "c4a03ead7f68e36e0958aa7204871ee6"
server_key_iv: "4df71ef46b902fb0e8a22eb43621a49c"
server_key_version: "encryptionkey@0"
I20260812 11:12:30.474476 8067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:30.494163 8067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:30.497709 8067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:30.498121 8067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:30.499699 8067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:12:30.500910 8067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:12:30.501011 8067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:30.501153 8067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:12:30.501204 8067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:30.511977 8067 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:46013
I20260812 11:12:30.512030 8258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:46013 every 8 connection(s)
I20260812 11:12:30.513034 8067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:12:30.517514 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 8067
Aug 12 11:12:30 dist-test-slave-drl0 krb5kdc[6636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533150, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:12:30.528893 8264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:30.518243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48072 (local address 127.17.44.254:36829)
0812 11:12:30.518508 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:30.518511 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:30.518686 (+ 175us) server_negotiation.cc:407] Connection header received
0812 11:12:30.518849 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:30.518852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:30.518891 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:30.519169 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:30.520141 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.521139 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.522889 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.523492 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.523678 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:30.525173 (+ 1495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:30.525179 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:30.525184 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:30.525216 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:30.526916 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:30.527469 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:30.527474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:30.527477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:30.527582 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:30.527950 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:30.527952 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:30.527954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:30.528142 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:30.528337 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:30.528528 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 11:12:30.528656 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":168,"server-negotiator.queue_time_us":209,"spinlock_wait_cycles":8960,"thread_start_us":99,"threads_started":1}
I20260812 11:12:30.530922 7861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:30.515161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:60085 (local address 127.17.44.254:36829)
0812 11:12:30.515299 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:30.515302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:30.516462 (+ 1160us) server_negotiation.cc:407] Connection header received
0812 11:12:30.518120 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:30.518124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:30.518184 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:30.518306 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:30.519997 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.521099 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.522947 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.523492 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.523676 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:30.526361 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:30.526364 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:30.526366 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:30.526390 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:30.528154 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:30.528909 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:30.528912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:30.528913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:30.529026 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:30.529540 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:30.529544 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:30.529546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:30.529787 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:30.529901 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:30.530676 (+ 775us) server_negotiation.cc:299] Negotiation successful
0812 11:12:30.530807 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":168,"server-negotiator.queue_time_us":50,"spinlock_wait_cycles":2304}
I20260812 11:12:30.531689 8262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:30.515512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.195:60085)
0812 11:12:30.516153 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:30.516195 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:30.517831 (+ 1636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:30.518520 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:30.518529 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:30.519085 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:30.519816 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:30.519837 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.521405 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.521409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:30.522823 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:30.522834 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.523702 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.523707 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:30.523916 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:30.524646 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:30.524662 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:30.526180 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:30.528304 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:30.528318 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:30.528336 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:30.528715 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:30.529142 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:30.529145 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:30.529147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:30.529306 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:30.529960 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:30.529969 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:30.530252 (+ 283us) client_negotiation.cc:769] Sending connection context
0812 11:12:30.530417 (+ 165us) client_negotiation.cc:240] Negotiation successful
0812 11:12:30.530716 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":140,"threads_started":1}
I20260812 11:12:30.532809 8259 heartbeater.cc:347] Connected to a master server at 127.17.44.254:36829
I20260812 11:12:30.533138 8259 heartbeater.cc:464] Registering TS with master...
I20260812 11:12:30.533660 8259 heartbeater.cc:511] Master 127.17.44.254:36829 requested a full tablet report, sending...
I20260812 11:12:30.534871 7554 ts_manager.cc:194] Registered new tserver with Master: 1ac60ba6fecc43bc856fae0dc226066e (127.17.44.195:46013)
I20260812 11:12:30.535730 7554 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.195@KRBTEST.COM'} at 127.17.44.195:60085
I20260812 11:12:30.544214 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:12:30.548875 7861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:30.544905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48074 (local address 127.17.44.254:36829)
0812 11:12:30.545057 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:30.545060 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:30.545260 (+ 200us) server_negotiation.cc:407] Connection header received
0812 11:12:30.545339 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:30.545341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:30.545466 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:30.545557 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:12:30.545870 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.546369 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.547321 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.547594 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.547731 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:30.548069 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:12:30.548481 (+ 412us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:12:30.548583 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:30.548719 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 11:12:30.548773 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 11:12:30.549505 7554 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:48074:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:12:30.552825 7554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:12:30.561810 7745 tablet_service.cc:1467] Processing CreateTablet for tablet 7603fe3770c94b1fb0190e4c3972888b (DEFAULT_TABLE table=test-table [id=762ab11de69e428e9b44f57530aed427]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:12:30.562505 7745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7603fe3770c94b1fb0190e4c3972888b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:12:30.565629 7680 tablet_bootstrap.cc:492] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9: Bootstrap starting.
I20260812 11:12:30.566329 7867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:30.560864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:46013 (local address 127.0.0.1:38968)
0812 11:12:30.561072 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:30.561088 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:30.561168 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:30.562237 (+ 1069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:30.562240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:30.562283 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:12:30.562452 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:30.562460 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.563884 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.563890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:30.564721 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:30.564731 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.565899 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.565903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:30.566004 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:30.566055 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 11:12:30.566105 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 11:12:30.566187 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":50,"spinlock_wait_cycles":27136}
I20260812 11:12:30.566653 7680 tablet_bootstrap.cc:654] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9: Neither blocks nor log segments found. Creating new log.
I20260812 11:12:30.566967 8272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:30.560925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:35327 (local address 127.0.0.1:40528)
0812 11:12:30.561219 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:30.561234 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:30.561324 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:30.562236 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:30.562240 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:30.562285 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:12:30.562620 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:30.562627 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.563969 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.563972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:30.565127 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:30.565140 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.566581 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.566586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:30.566670 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:30.566691 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 11:12:30.566741 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 11:12:30.566810 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":177,"spinlock_wait_cycles":4736,"thread_start_us":121,"threads_started":1}
I20260812 11:12:30.567232 8273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:30.561351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38968 (local address 127.17.44.195:46013)
0812 11:12:30.561674 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:30.561687 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:30.561712 (+ 25us) server_negotiation.cc:407] Connection header received
0812 11:12:30.561793 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:30.561799 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:30.562057 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:30.562362 (+ 305us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:12:30.562566 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.563732 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.564865 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.565744 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.566621 (+ 877us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:30.566777 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:30.566842 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 11:12:30.566960 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":105,"threads_started":1}
I20260812 11:12:30.567610 8274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:30.561383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40528 (local address 127.17.44.194:35327)
0812 11:12:30.561771 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:30.561780 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:30.561799 (+ 19us) server_negotiation.cc:407] Connection header received
0812 11:12:30.561865 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:30.561869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:30.562054 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:30.562453 (+ 399us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:12:30.562741 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.563761 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.565274 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.566429 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.567152 (+ 723us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:30.567295 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:30.567371 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 11:12:30.567464 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":122,"threads_started":1}
I20260812 11:12:30.568388 7680 tablet_bootstrap.cc:492] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9: No bootstrap required, opened a new log
I20260812 11:12:30.568477 7680 ts_tablet_manager.cc:1397] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 11:12:30.568920 7680 raft_consensus.cc:348] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } }
I20260812 11:12:30.569168 7680 raft_consensus.cc:374] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:12:30.569365 7680 raft_consensus.cc:729] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bcc97a954b4d69888b38749ee663c9, State: Initialized, Role: FOLLOWER
I20260812 11:12:30.569592 7680 consensus_queue.cc:260] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } }
I20260812 11:12:30.570006 7680 ts_tablet_manager.cc:1428] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:12:30.570480 8158 tablet_service.cc:1467] Processing CreateTablet for tablet 7603fe3770c94b1fb0190e4c3972888b (DEFAULT_TABLE table=test-table [id=762ab11de69e428e9b44f57530aed427]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:12:30.570473 7970 tablet_service.cc:1467] Processing CreateTablet for tablet 7603fe3770c94b1fb0190e4c3972888b (DEFAULT_TABLE table=test-table [id=762ab11de69e428e9b44f57530aed427]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:12:30.571565 7970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7603fe3770c94b1fb0190e4c3972888b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:12:30.571584 8158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7603fe3770c94b1fb0190e4c3972888b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:12:30.578866 8277 tablet_bootstrap.cc:492] T 7603fe3770c94b1fb0190e4c3972888b P 1ac60ba6fecc43bc856fae0dc226066e: Bootstrap starting.
I20260812 11:12:30.579617 8279 tablet_bootstrap.cc:492] T 7603fe3770c94b1fb0190e4c3972888b P f8b256eb81ee4ceb93c293d70db13348: Bootstrap starting.
I20260812 11:12:30.581871 8277 tablet_bootstrap.cc:654] T 7603fe3770c94b1fb0190e4c3972888b P 1ac60ba6fecc43bc856fae0dc226066e: Neither blocks nor log segments found. Creating new log.
I20260812 11:12:30.582165 8279 tablet_bootstrap.cc:654] T 7603fe3770c94b1fb0190e4c3972888b P f8b256eb81ee4ceb93c293d70db13348: Neither blocks nor log segments found. Creating new log.
I20260812 11:12:30.583096 8279 log.cc:824] T 7603fe3770c94b1fb0190e4c3972888b P f8b256eb81ee4ceb93c293d70db13348: Log is configured to *not* fsync() on all Append() calls
I20260812 11:12:30.583096 8277 log.cc:824] T 7603fe3770c94b1fb0190e4c3972888b P 1ac60ba6fecc43bc856fae0dc226066e: Log is configured to *not* fsync() on all Append() calls
I20260812 11:12:30.585485 8277 tablet_bootstrap.cc:492] T 7603fe3770c94b1fb0190e4c3972888b P 1ac60ba6fecc43bc856fae0dc226066e: No bootstrap required, opened a new log
I20260812 11:12:30.585667 8279 tablet_bootstrap.cc:492] T 7603fe3770c94b1fb0190e4c3972888b P f8b256eb81ee4ceb93c293d70db13348: No bootstrap required, opened a new log
I20260812 11:12:30.585690 8277 ts_tablet_manager.cc:1397] T 7603fe3770c94b1fb0190e4c3972888b P 1ac60ba6fecc43bc856fae0dc226066e: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260812 11:12:30.585848 8279 ts_tablet_manager.cc:1397] T 7603fe3770c94b1fb0190e4c3972888b P f8b256eb81ee4ceb93c293d70db13348: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
W20260812 11:12:30.586766 7860 tablet.cc:2369] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:12:30.589644 8277 raft_consensus.cc:348] T 7603fe3770c94b1fb0190e4c3972888b P 1ac60ba6fecc43bc856fae0dc226066e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } }
I20260812 11:12:30.589644 8279 raft_consensus.cc:348] T 7603fe3770c94b1fb0190e4c3972888b P f8b256eb81ee4ceb93c293d70db13348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } }
I20260812 11:12:30.589922 8277 raft_consensus.cc:374] T 7603fe3770c94b1fb0190e4c3972888b P 1ac60ba6fecc43bc856fae0dc226066e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:12:30.589922 8279 raft_consensus.cc:374] T 7603fe3770c94b1fb0190e4c3972888b P f8b256eb81ee4ceb93c293d70db13348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:12:30.589977 8279 raft_consensus.cc:729] T 7603fe3770c94b1fb0190e4c3972888b P f8b256eb81ee4ceb93c293d70db13348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8b256eb81ee4ceb93c293d70db13348, State: Initialized, Role: FOLLOWER
I20260812 11:12:30.589977 8277 raft_consensus.cc:729] T 7603fe3770c94b1fb0190e4c3972888b P 1ac60ba6fecc43bc856fae0dc226066e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ac60ba6fecc43bc856fae0dc226066e, State: Initialized, Role: FOLLOWER
I20260812 11:12:30.590435 8279 consensus_queue.cc:260] T 7603fe3770c94b1fb0190e4c3972888b P f8b256eb81ee4ceb93c293d70db13348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } }
I20260812 11:12:30.590435 8277 consensus_queue.cc:260] T 7603fe3770c94b1fb0190e4c3972888b P 1ac60ba6fecc43bc856fae0dc226066e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } }
I20260812 11:12:30.591185 8259 heartbeater.cc:503] Master 127.17.44.254:36829 was elected leader, sending a full tablet report...
I20260812 11:12:30.591461 8277 ts_tablet_manager.cc:1428] T 7603fe3770c94b1fb0190e4c3972888b P 1ac60ba6fecc43bc856fae0dc226066e: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260812 11:12:30.591461 8279 ts_tablet_manager.cc:1428] T 7603fe3770c94b1fb0190e4c3972888b P f8b256eb81ee4ceb93c293d70db13348: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:12:30.591790 8064 heartbeater.cc:503] Master 127.17.44.254:36829 was elected leader, sending a full tablet report...
I20260812 11:12:30.680295 7865 raft_consensus.cc:482] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:12:30.680662 7865 raft_consensus.cc:504] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } }
I20260812 11:12:30.681849 7865 leader_election.cc:288] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ac60ba6fecc43bc856fae0dc226066e (127.17.44.195:46013), f8b256eb81ee4ceb93c293d70db13348 (127.17.44.194:35327)
I20260812 11:12:30.687812 8274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:30.682265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:36077 (local address 127.17.44.194:35327)
0812 11:12:30.682528 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:30.682534 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:30.682553 (+ 19us) server_negotiation.cc:407] Connection header received
0812 11:12:30.682851 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:30.682854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:30.682909 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:30.683033 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:12:30.683614 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.684202 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.686248 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.686901 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.687058 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:30.687111 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:30.687602 (+ 491us) server_negotiation.cc:299] Negotiation successful
0812 11:12:30.687680 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260812 11:12:30.687812 8273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:30.682343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:32947 (local address 127.17.44.195:46013)
0812 11:12:30.682478 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:30.682483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:30.682607 (+ 124us) server_negotiation.cc:407] Connection header received
0812 11:12:30.682835 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:30.682839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:30.682902 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:30.683033 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:12:30.683648 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.684243 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.686249 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.686901 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.687057 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:30.687111 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:30.687602 (+ 491us) server_negotiation.cc:299] Negotiation successful
0812 11:12:30.687680 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 11:12:30.688031 8289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:30.682162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:35327 (local address 127.17.44.193:36077)
0812 11:12:30.682479 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:30.682495 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:30.682620 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:30.683222 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:30.683229 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:30.683256 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:12:30.683450 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:30.683457 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.684366 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.684370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:30.686078 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:30.686090 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.687094 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.687098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:30.687323 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:30.687382 (+ 59us) client_negotiation.cc:769] Sending connection context
0812 11:12:30.687552 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 11:12:30.687704 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":362,"thread_start_us":109,"threads_started":1}
I20260812 11:12:30.688040 8288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:30.682206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:46013 (local address 127.17.44.193:32947)
0812 11:12:30.682518 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:30.682532 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:30.682620 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:30.683154 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:30.683162 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:30.683218 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:12:30.683427 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:30.683435 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.684417 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.684420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:30.686078 (+ 1658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:30.686090 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.687094 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.687098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:30.687275 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:30.687379 (+ 104us) client_negotiation.cc:769] Sending connection context
0812 11:12:30.687552 (+ 173us) client_negotiation.cc:240] Negotiation successful
0812 11:12:30.687704 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":268,"spinlock_wait_cycles":2048,"thread_start_us":89,"threads_started":1}
I20260812 11:12:30.688900 8189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7603fe3770c94b1fb0190e4c3972888b" candidate_uuid: "e9bcc97a954b4d69888b38749ee663c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ac60ba6fecc43bc856fae0dc226066e" is_pre_election: true
I20260812 11:12:30.688896 8000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7603fe3770c94b1fb0190e4c3972888b" candidate_uuid: "e9bcc97a954b4d69888b38749ee663c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8b256eb81ee4ceb93c293d70db13348" is_pre_election: true
I20260812 11:12:30.689448 8189 raft_consensus.cc:2454] T 7603fe3770c94b1fb0190e4c3972888b P 1ac60ba6fecc43bc856fae0dc226066e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9bcc97a954b4d69888b38749ee663c9 in term 0.
I20260812 11:12:30.689448 8000 raft_consensus.cc:2454] T 7603fe3770c94b1fb0190e4c3972888b P f8b256eb81ee4ceb93c293d70db13348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9bcc97a954b4d69888b38749ee663c9 in term 0.
I20260812 11:12:30.690213 7673 leader_election.cc:302] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1ac60ba6fecc43bc856fae0dc226066e, e9bcc97a954b4d69888b38749ee663c9, f8b256eb81ee4ceb93c293d70db13348; no voters:
I20260812 11:12:30.690486 7865 raft_consensus.cc:2781] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:12:30.690573 7865 raft_consensus.cc:482] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:12:30.690618 7865 raft_consensus.cc:3037] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:12:30.691664 7865 raft_consensus.cc:504] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } }
I20260812 11:12:30.692023 7865 leader_election.cc:288] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [CANDIDATE]: Term 1 election: Requested vote from peers 1ac60ba6fecc43bc856fae0dc226066e (127.17.44.195:46013), f8b256eb81ee4ceb93c293d70db13348 (127.17.44.194:35327)
I20260812 11:12:30.692416 8189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7603fe3770c94b1fb0190e4c3972888b" candidate_uuid: "e9bcc97a954b4d69888b38749ee663c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ac60ba6fecc43bc856fae0dc226066e"
I20260812 11:12:30.692565 8189 raft_consensus.cc:3037] T 7603fe3770c94b1fb0190e4c3972888b P 1ac60ba6fecc43bc856fae0dc226066e [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:12:30.692616 8000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7603fe3770c94b1fb0190e4c3972888b" candidate_uuid: "e9bcc97a954b4d69888b38749ee663c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8b256eb81ee4ceb93c293d70db13348"
I20260812 11:12:30.692765 8000 raft_consensus.cc:3037] T 7603fe3770c94b1fb0190e4c3972888b P f8b256eb81ee4ceb93c293d70db13348 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:12:30.693692 8189 raft_consensus.cc:2454] T 7603fe3770c94b1fb0190e4c3972888b P 1ac60ba6fecc43bc856fae0dc226066e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9bcc97a954b4d69888b38749ee663c9 in term 1.
I20260812 11:12:30.693945 8000 raft_consensus.cc:2454] T 7603fe3770c94b1fb0190e4c3972888b P f8b256eb81ee4ceb93c293d70db13348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9bcc97a954b4d69888b38749ee663c9 in term 1.
I20260812 11:12:30.694288 7673 leader_election.cc:302] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ac60ba6fecc43bc856fae0dc226066e, e9bcc97a954b4d69888b38749ee663c9; no voters:
I20260812 11:12:30.695155 7865 raft_consensus.cc:2781] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:12:30.695513 7865 raft_consensus.cc:686] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 1 LEADER]: Becoming Leader. State: Replica: e9bcc97a954b4d69888b38749ee663c9, State: Running, Role: LEADER
I20260812 11:12:30.695873 7865 consensus_queue.cc:237] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } }
I20260812 11:12:30.699812 7552 catalog_manager.cc:5495] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e9bcc97a954b4d69888b38749ee663c9 (127.17.44.193). New cstate: current_term: 1 leader_uuid: "e9bcc97a954b4d69888b38749ee663c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } health_report { overall_health: HEALTHY } } }
I20260812 11:12:30.726897 8294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:30.722154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34620 (local address 127.17.44.193:45743)
0812 11:12:30.722737 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:30.722748 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:30.722761 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:12:30.722902 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:30.722905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:30.723029 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:30.723284 (+ 255us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:12:30.723739 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.724352 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.725368 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:30.725584 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:30.725673 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:30.725992 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:12:30.726464 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:12:30.726554 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:30.726712 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 11:12:30.726772 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":141,"threads_started":1}
I20260812 11:12:30.731141 8000 raft_consensus.cc:1261] T 7603fe3770c94b1fb0190e4c3972888b P f8b256eb81ee4ceb93c293d70db13348 [term 1 FOLLOWER]: Refusing update from remote peer e9bcc97a954b4d69888b38749ee663c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:12:30.731142 8189 raft_consensus.cc:1261] T 7603fe3770c94b1fb0190e4c3972888b P 1ac60ba6fecc43bc856fae0dc226066e [term 1 FOLLOWER]: Refusing update from remote peer e9bcc97a954b4d69888b38749ee663c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:12:30.731879 8290 consensus_queue.cc:1035] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:12:30.732123 7865 consensus_queue.cc:1035] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:12:30.742746 8298 mvcc.cc:205] Tried to move back new op lower bound from 7317639785387827200 to 7317639785253974016. Current Snapshot: MvccSnapshot[applied={T|T < 7317639785387827200 or (T in {7317639785387827200})}]
I20260812 11:12:30.744100 8297 mvcc.cc:205] Tried to move back new op lower bound from 7317639785387827200 to 7317639785253974016. Current Snapshot: MvccSnapshot[applied={T|T < 7317639785387827200}]
I20260812 11:12:30.749243 7552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:48074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:12:30.749432 7552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:48074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:12:30.752194 7552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb: Sending DeleteTablet for 3 replicas of tablet 7603fe3770c94b1fb0190e4c3972888b
I20260812 11:12:30.752919 7745 tablet_service.cc:1514] Processing DeleteTablet for tablet 7603fe3770c94b1fb0190e4c3972888b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:12:30 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:34604
I20260812 11:12:30.752970 7970 tablet_service.cc:1514] Processing DeleteTablet for tablet 7603fe3770c94b1fb0190e4c3972888b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:12:30 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:40528
I20260812 11:12:30.752987 8158 tablet_service.cc:1514] Processing DeleteTablet for tablet 7603fe3770c94b1fb0190e4c3972888b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:12:30 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:38968
I20260812 11:12:30.753549 8310 tablet_replica.cc:330] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9: stopping tablet replica
I20260812 11:12:30.753680 8310 raft_consensus.cc:2229] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:12:30.753683 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 7612
I20260812 11:12:30.753726 8312 tablet_replica.cc:330] T 7603fe3770c94b1fb0190e4c3972888b P 1ac60ba6fecc43bc856fae0dc226066e: stopping tablet replica
I20260812 11:12:30.754202 8312 raft_consensus.cc:2229] T 7603fe3770c94b1fb0190e4c3972888b P 1ac60ba6fecc43bc856fae0dc226066e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:12:30.756974 8312 raft_consensus.cc:2258] T 7603fe3770c94b1fb0190e4c3972888b P 1ac60ba6fecc43bc856fae0dc226066e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:12:30.759511 8314 tablet_replica.cc:330] T 7603fe3770c94b1fb0190e4c3972888b P f8b256eb81ee4ceb93c293d70db13348: stopping tablet replica
I20260812 11:12:30.759815 8314 raft_consensus.cc:2229] T 7603fe3770c94b1fb0190e4c3972888b P f8b256eb81ee4ceb93c293d70db13348 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:12:30.760464 8314 raft_consensus.cc:2258] T 7603fe3770c94b1fb0190e4c3972888b P f8b256eb81ee4ceb93c293d70db13348 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:12:30.760794 8312 ts_tablet_manager.cc:1905] T 7603fe3770c94b1fb0190e4c3972888b P 1ac60ba6fecc43bc856fae0dc226066e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:12:30.764159 8312 ts_tablet_manager.cc:1918] T 7603fe3770c94b1fb0190e4c3972888b P 1ac60ba6fecc43bc856fae0dc226066e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:12:30.764300 8312 log.cc:1196] T 7603fe3770c94b1fb0190e4c3972888b P 1ac60ba6fecc43bc856fae0dc226066e: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/7603fe3770c94b1fb0190e4c3972888b
I20260812 11:12:30.764475 8314 ts_tablet_manager.cc:1905] T 7603fe3770c94b1fb0190e4c3972888b P f8b256eb81ee4ceb93c293d70db13348: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:12:30.764742 8312 ts_tablet_manager.cc:1939] T 7603fe3770c94b1fb0190e4c3972888b P 1ac60ba6fecc43bc856fae0dc226066e: Deleting consensus metadata
I20260812 11:12:30.766121 7522 catalog_manager.cc:4841] TS 1ac60ba6fecc43bc856fae0dc226066e (127.17.44.195:46013): tablet 7603fe3770c94b1fb0190e4c3972888b (table test-table [id=762ab11de69e428e9b44f57530aed427]) successfully deleted
I20260812 11:12:30.767131 8314 ts_tablet_manager.cc:1918] T 7603fe3770c94b1fb0190e4c3972888b P f8b256eb81ee4ceb93c293d70db13348: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:12:30.767230 8314 log.cc:1196] T 7603fe3770c94b1fb0190e4c3972888b P f8b256eb81ee4ceb93c293d70db13348: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/7603fe3770c94b1fb0190e4c3972888b
I20260812 11:12:30.767560 8314 ts_tablet_manager.cc:1939] T 7603fe3770c94b1fb0190e4c3972888b P f8b256eb81ee4ceb93c293d70db13348: Deleting consensus metadata
I20260812 11:12:30.768728 7526 catalog_manager.cc:4841] TS f8b256eb81ee4ceb93c293d70db13348 (127.17.44.194:35327): tablet 7603fe3770c94b1fb0190e4c3972888b (table test-table [id=762ab11de69e428e9b44f57530aed427]) successfully deleted
W20260812 11:12:30.770676 7525 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.44.193:45743 (error 108)
I20260812 11:12:30.770721 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 7862
I20260812 11:12:30.780191 8272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:30.779801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:45743 (local address 127.0.0.1:34634)
0812 11:12:30.779990 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:30.780077 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.193:45743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 11:12:30.780396 7525 catalog_manager.cc:4584] TS e9bcc97a954b4d69888b38749ee663c9 (127.17.44.193:45743): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7603fe3770c94b1fb0190e4c3972888b: Network error: Client connection negotiation failed: client connection to 127.17.44.193:45743: connect: Connection refused (error 111)
I20260812 11:12:30.786033 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 8067
I20260812 11:12:30.802031 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 7496
I20260812 11:12:30.815136 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:36829
--webserver_interface=127.17.44.254
--webserver_port=43893
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:46029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:36829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:12:30.941059 8316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:30.941326 8316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:30.941370 8316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:30.945775 8316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:12:30.945844 8316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:30.945866 8316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:30.945883 8316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:12:30.945933 8316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:12:30.951548 8316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:46029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:36829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:36829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=43893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:30.952642 8316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:30.953689 8316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:30.963786 8327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:30.963886 8323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:30.963771 8322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:30.964071 8316 server_base.cc:1034] running on GCE node
I20260812 11:12:30.964994 8316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:30.966233 8316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:30.967473 8316 hybrid_clock.cc:648] HybridClock initialized: now 1786533150967447 us; error 48 us; skew 500 ppm
Aug 12 11:12:30 dist-test-slave-drl0 krb5kdc[6636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533150, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:30.970449 8316 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:12:30.971469 8316 webserver.cc:460] Webserver started at http://127.17.44.254:43893/ using document root <none> and password file <none>
I20260812 11:12:30.972009 8316 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:30.972070 8316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:12:30.976341 8316 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:12:30.978842 8336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:30.980270 8316 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 11:12:30.980387 8316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "b11244d9a967487ba96bdcf19d8b68eb"
format_stamp: "Formatted at 2026-08-12 11:12:28 on dist-test-slave-drl0"
server_key: "94d5f5d03ebfa615a0832c5d8962236d"
server_key_iv: "a6015e6f897679ff3d48f6bbb7b0bac7"
server_key_version: "encryptionkey@0"
I20260812 11:12:30.980924 8316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:30.995224 8316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:30.998358 8316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:30.998733 8316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:31.007596 8316 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:36829
I20260812 11:12:31.007678 8421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:36829 every 8 connection(s)
I20260812 11:12:31.008724 8316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:12:31.013113 8422 sys_catalog.cc:263] Verifying existing consensus state
I20260812 11:12:31.014060 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 8316
I20260812 11:12:31.014533 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:45743
--local_ip_for_outbound_sockets=127.17.44.193
--tserver_master_addrs=127.17.44.254:36829
--webserver_port=46773
--webserver_interface=127.17.44.193
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:46029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:12:31.015375 8422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb: Bootstrap starting.
I20260812 11:12:31.041970 8422 log.cc:824] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb: Log is configured to *not* fsync() on all Append() calls
I20260812 11:12:31.056177 8422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 11:12:31.056604 8422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb: Bootstrap complete.
I20260812 11:12:31.061231 8422 raft_consensus.cc:348] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b11244d9a967487ba96bdcf19d8b68eb" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36829 } }
I20260812 11:12:31.061681 8422 raft_consensus.cc:729] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b11244d9a967487ba96bdcf19d8b68eb, State: Initialized, Role: FOLLOWER
I20260812 11:12:31.062259 8422 consensus_queue.cc:260] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b11244d9a967487ba96bdcf19d8b68eb" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36829 } }
I20260812 11:12:31.062410 8422 raft_consensus.cc:388] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:12:31.062507 8422 raft_consensus.cc:482] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:12:31.062630 8422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 2 FOLLOWER]: Advancing to term 3
I20260812 11:12:31.064693 8422 raft_consensus.cc:504] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b11244d9a967487ba96bdcf19d8b68eb" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36829 } }
I20260812 11:12:31.065058 8422 leader_election.cc:302] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b11244d9a967487ba96bdcf19d8b68eb; no voters:
I20260812 11:12:31.065385 8422 leader_election.cc:288] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 11:12:31.065528 8428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 3 FOLLOWER]: Leader election won for term 3
I20260812 11:12:31.065802 8428 raft_consensus.cc:686] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [term 3 LEADER]: Becoming Leader. State: Replica: b11244d9a967487ba96bdcf19d8b68eb, State: Running, Role: LEADER
I20260812 11:12:31.066118 8428 consensus_queue.cc:237] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b11244d9a967487ba96bdcf19d8b68eb" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36829 } }
I20260812 11:12:31.066636 8422 sys_catalog.cc:564] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:12:31.067752 8430 sys_catalog.cc:455] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b11244d9a967487ba96bdcf19d8b68eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b11244d9a967487ba96bdcf19d8b68eb" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36829 } } }
I20260812 11:12:31.067893 8430 sys_catalog.cc:458] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [sys.catalog]: This master's current role is: LEADER
I20260812 11:12:31.067776 8432 sys_catalog.cc:455] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader b11244d9a967487ba96bdcf19d8b68eb. Latest consensus state: current_term: 3 leader_uuid: "b11244d9a967487ba96bdcf19d8b68eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b11244d9a967487ba96bdcf19d8b68eb" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36829 } } }
I20260812 11:12:31.068252 8432 sys_catalog.cc:458] T 00000000000000000000000000000000 P b11244d9a967487ba96bdcf19d8b68eb [sys.catalog]: This master's current role is: LEADER
I20260812 11:12:31.068312 8441 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:12:31.074456 8441 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:12:31.075196 8441 catalog_manager.cc:1259] Loaded cluster ID: 9d7a0d22bad647f381a1ec834f08f692
I20260812 11:12:31.075263 8441 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 11:12:31.076285 8441 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f7c27e919ad google::LogMessage::Fail() at ??:0
@ 0x7f7c27e9488a google::LogMessage::SendToLog() at ??:0
@ 0x7f7c27e914de google::LogMessage::Flush() at ??:0
@ 0x7f7c27e92a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f7c327d0fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f7c327d1d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f7c327ce579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f7c32812640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f7c2a0e33d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f7c2a0e3c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f7c2a0e53f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f7c2a0d4512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f7c2b7186db start_thread at ??:0
@ 0x7f7c2698f61f clone at ??:0
W20260812 11:12:31.192751 8424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:31.193136 8424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:31.193203 8424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:31.199689 8424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:31.199771 8424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:31.199872 8424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:12:31.207758 8424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:46029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:45743
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=46773
--tserver_master_addrs=127.17.44.254:36829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:31.209108 8424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:31.210777 8424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:31.224231 8477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:31.224577 8482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:31.224745 8476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:31.224784 8424 server_base.cc:1034] running on GCE node
I20260812 11:12:31.225178 8424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:31.225829 8424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:31.227017 8424 hybrid_clock.cc:648] HybridClock initialized: now 1786533151226972 us; error 62 us; skew 500 ppm
Aug 12 11:12:31 dist-test-slave-drl0 krb5kdc[6636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533151, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:31.231312 8424 init.cc:377] Logged in from keytab as kudu/127.17.44.193@KRBTEST.COM (short username kudu)
I20260812 11:12:31.232715 8424 webserver.cc:460] Webserver started at http://127.17.44.193:46773/ using document root <none> and password file <none>
I20260812 11:12:31.233486 8424 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:31.233639 8424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:12:31.238654 8424 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:12:31.241252 8500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:31.243206 8424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 11:12:31.243358 8424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "e9bcc97a954b4d69888b38749ee663c9"
format_stamp: "Formatted at 2026-08-12 11:12:28 on dist-test-slave-drl0"
server_key: "5106f05786ec90df4275b3ed6dd1620f"
server_key_iv: "64ad37224cdca5a9226a5e153b457684"
server_key_version: "encryptionkey@0"
I20260812 11:12:31.243979 8424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:31.270804 8424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:31.274238 8424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:31.274451 8424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:31.275225 8424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:12:31.276453 8509 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 11:12:31.282922 8424 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 11:12:31.283077 8424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.000s
I20260812 11:12:31.283162 8424 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 11:12:31.284950 8424 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 11:12:31.285034 8424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 11:12:31.285318 8509 tablet_bootstrap.cc:492] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9: Bootstrap starting.
I20260812 11:12:31.296934 8424 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:45743
I20260812 11:12:31.297078 8675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:45743 every 8 connection(s)
I20260812 11:12:31.298233 8424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:12:31.304910 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 8424
I20260812 11:12:31.305399 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:35327
--local_ip_for_outbound_sockets=127.17.44.194
--tserver_master_addrs=127.17.44.254:36829
--webserver_port=37751
--webserver_interface=127.17.44.194
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:46029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:12:31.315223 8509 log.cc:824] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9: Log is configured to *not* fsync() on all Append() calls
I20260812 11:12:31.322402 8509 tablet_bootstrap.cc:492] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 11:12:31.322816 8509 tablet_bootstrap.cc:492] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9: Bootstrap complete.
I20260812 11:12:31.324069 8509 ts_tablet_manager.cc:1397] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9: Time spent bootstrapping tablet: real 0.039s user 0.019s sys 0.015s
I20260812 11:12:31.326714 8509 raft_consensus.cc:348] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } }
I20260812 11:12:31.327392 8509 raft_consensus.cc:729] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bcc97a954b4d69888b38749ee663c9, State: Initialized, Role: FOLLOWER
I20260812 11:12:31.327935 8509 consensus_queue.cc:260] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } }
I20260812 11:12:31.328898 8509 ts_tablet_manager.cc:1428] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260812 11:12:31.387037 8681 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:12:31.300700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.193:54839)
0812 11:12:31.301544 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:31.301726 (+ 182us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:31.302664 (+ 938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:31.386077 (+ 83413us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":326,"threads_started":1}
W20260812 11:12:31.387616 8676 heartbeater.cc:644] Failed to heartbeat to 127.17.44.254:36829 (0 consecutive failures): Network error: Failed to ping master at 127.17.44.254:36829: Client connection negotiation failed: client connection to 127.17.44.254:36829: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 11:12:31.388351 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:31.388091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.193:59045)
0812 11:12:31.388229 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:31.388300 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 11:12:31.388969 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:31.388756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.193:40289)
0812 11:12:31.388869 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:31.388914 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 11:12:31.389132 8676 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 11:12:31.446789 8682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:31.447131 8682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:31.447218 8682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:31.451655 8682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:31.451740 8682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:31.451833 8682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:12:31.457279 8682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:46029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:35327
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=37751
--tserver_master_addrs=127.17.44.254:36829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:31.458295 8682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:31.459365 8682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:31.469089 8698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:31.469187 8700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:31.469234 8697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:31.469348 8682 server_base.cc:1034] running on GCE node
I20260812 11:12:31.469911 8682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:31.470448 8682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:31.471648 8682 hybrid_clock.cc:648] HybridClock initialized: now 1786533151471633 us; error 35 us; skew 500 ppm
Aug 12 11:12:31 dist-test-slave-drl0 krb5kdc[6636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533151, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:31.474752 8682 init.cc:377] Logged in from keytab as kudu/127.17.44.194@KRBTEST.COM (short username kudu)
I20260812 11:12:31.475662 8682 webserver.cc:460] Webserver started at http://127.17.44.194:37751/ using document root <none> and password file <none>
I20260812 11:12:31.476254 8682 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:31.476325 8682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:12:31.480168 8682 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.003s
I20260812 11:12:31.482226 8713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:31.483572 8682 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:12:31.483711 8682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "f8b256eb81ee4ceb93c293d70db13348"
format_stamp: "Formatted at 2026-08-12 11:12:29 on dist-test-slave-drl0"
server_key: "32b126ae723620c2280d54cacf8efdc5"
server_key_iv: "f6167c2544a70885f2ce1da869d654cd"
server_key_version: "encryptionkey@0"
I20260812 11:12:31.484292 8682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:31.503716 8682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:31.507099 8682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:31.507323 8682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:31.507944 8682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:12:31.508782 8682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:12:31.508829 8682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:31.508870 8682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:12:31.508927 8682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:31.518906 8682 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:35327
I20260812 11:12:31.519008 8882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:35327 every 8 connection(s)
I20260812 11:12:31.520100 8682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:12:31.525657 8886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:31.523432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.194:44965)
0812 11:12:31.524241 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:31.524367 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":263,"threads_started":1}
W20260812 11:12:31.526275 8883 heartbeater.cc:644] Failed to heartbeat to 127.17.44.254:36829 (0 consecutive failures): Network error: Failed to ping master at 127.17.44.254:36829: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
I20260812 11:12:31.527195 8886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:31.526918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.194:45591)
0812 11:12:31.527076 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:31.527138 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260812 11:12:31.528080 8886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:31.527851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.194:39529)
0812 11:12:31.527982 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:31.528029 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 11:12:31.528311 8883 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 11:12:31.530354 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 8682
I20260812 11:12:31.530875 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:46013
--local_ip_for_outbound_sockets=127.17.44.195
--tserver_master_addrs=127.17.44.254:36829
--webserver_port=35077
--webserver_interface=127.17.44.195
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:46029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:12:31.653136 8887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:31.653396 8887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:31.653441 8887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:31.657640 8887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:31.657697 8887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:31.657756 8887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:12:31.663739 8887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:46029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:46013
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=35077
--tserver_master_addrs=127.17.44.254:36829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:31.664777 8887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:31.666141 8887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:31.677768 8900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:31.677778 8897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:31.677760 8898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:31.678395 8887 server_base.cc:1034] running on GCE node
I20260812 11:12:31.678854 8887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:31.679911 8887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:31.681143 8887 hybrid_clock.cc:648] HybridClock initialized: now 1786533151681124 us; error 45 us; skew 500 ppm
Aug 12 11:12:31 dist-test-slave-drl0 krb5kdc[6636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533151, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:31.684244 8887 init.cc:377] Logged in from keytab as kudu/127.17.44.195@KRBTEST.COM (short username kudu)
I20260812 11:12:31.685092 8887 webserver.cc:460] Webserver started at http://127.17.44.195:35077/ using document root <none> and password file <none>
I20260812 11:12:31.685652 8887 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:31.685714 8887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:12:31.690510 8887 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:12:31.693301 8910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:31.694769 8887 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:12:31.694877 8887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "1ac60ba6fecc43bc856fae0dc226066e"
format_stamp: "Formatted at 2026-08-12 11:12:29 on dist-test-slave-drl0"
server_key: "c4a03ead7f68e36e0958aa7204871ee6"
server_key_iv: "4df71ef46b902fb0e8a22eb43621a49c"
server_key_version: "encryptionkey@0"
I20260812 11:12:31.695650 8887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:31.710345 8887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:31.713932 8887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:31.714243 8887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:31.714881 8887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:12:31.715839 8887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:12:31.715914 8887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:31.715983 8887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:12:31.716032 8887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:31.728449 8887 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:46013
I20260812 11:12:31.728549 9086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:46013 every 8 connection(s)
I20260812 11:12:31.729516 8887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:12:31.733556 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:31.731913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.195:39725)
0812 11:12:31.732425 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:31.732877 (+ 452us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":130,"threads_started":1}
W20260812 11:12:31.734265 9087 heartbeater.cc:644] Failed to heartbeat to 127.17.44.254:36829 (0 consecutive failures): Network error: Failed to ping master at 127.17.44.254:36829: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
I20260812 11:12:31.735109 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:31.734654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.195:35963)
0812 11:12:31.734881 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:31.735007 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260812 11:12:31.735947 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:31.735690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.195:48705)
0812 11:12:31.735824 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:31.735875 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 11:12:31.736299 9087 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 11:12:31.739630 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 8887
I20260812 11:12:32.390444 9092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:32.389799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.193:53199)
0812 11:12:32.390172 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:32.390286 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":194,"threads_started":1}
I20260812 11:12:32.529321 9093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:32.528881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.194:59513)
0812 11:12:32.529154 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:32.529219 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":77,"threads_started":1}
I20260812 11:12:32.684916 9094 raft_consensus.cc:482] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:12:32.685151 9094 raft_consensus.cc:504] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } }
I20260812 11:12:32.686376 9094 leader_election.cc:288] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ac60ba6fecc43bc856fae0dc226066e (127.17.44.195:46013), f8b256eb81ee4ceb93c293d70db13348 (127.17.44.194:35327)
Aug 12 11:12:32 dist-test-slave-drl0 krb5kdc[6636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533151, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.194@KRBTEST.COM
Aug 12 11:12:32 dist-test-slave-drl0 krb5kdc[6636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533151, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.195@KRBTEST.COM
I20260812 11:12:32.712004 9097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:32.687268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:42229 (local address 127.17.44.194:35327)
0812 11:12:32.687823 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:32.687880 (+ 57us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:32.687952 (+ 72us) server_negotiation.cc:407] Connection header received
0812 11:12:32.689303 (+ 1351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:32.689314 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:32.689710 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:32.690060 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:32.691583 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:32.693583 (+ 2000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:32.695859 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:32.696721 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:32.696909 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:32.705919 (+ 9010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:32.705931 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:32.705945 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:32.705975 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:32.707907 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:32.708722 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:32.708732 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:32.708741 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:32.708894 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:32.709714 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:32.709719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:32.709721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:32.710438 (+ 717us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:32.710690 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:32.711566 (+ 876us) server_negotiation.cc:299] Negotiation successful
0812 11:12:32.711838 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":106,"threads_started":1}
I20260812 11:12:32.712165 9095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:32.686699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:35327 (local address 127.17.44.193:42229)
0812 11:12:32.687079 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:32.687095 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:32.687231 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:32.689994 (+ 2763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:32.689997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:32.690611 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:32.691240 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:32.691338 (+ 98us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:32.693811 (+ 2473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:32.693815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:32.695670 (+ 1855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:32.695682 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:32.696912 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:32.696916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:32.697375 (+ 459us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:32.703062 (+ 5687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:32.703093 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:32.705628 (+ 2535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:32.708044 (+ 2416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:32.708051 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:32.708064 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:32.708529 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:32.709153 (+ 624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:32.709159 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:32.709161 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:32.709475 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:32.710597 (+ 1122us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:32.710606 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:32.710953 (+ 347us) client_negotiation.cc:769] Sending connection context
0812 11:12:32.711386 (+ 433us) client_negotiation.cc:240] Negotiation successful
0812 11:12:32.711995 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":451,"spinlock_wait_cycles":45952,"thread_start_us":58,"threads_started":1}
I20260812 11:12:32.712167 9092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:32.686588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:46013 (local address 127.17.44.193:48343)
0812 11:12:32.686732 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:32.686751 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:32.687143 (+ 392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:32.689878 (+ 2735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:32.689885 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:32.690554 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:32.691248 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:32.691288 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:32.693811 (+ 2523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:32.693815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:32.695670 (+ 1855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:32.695682 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:32.696912 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:32.696917 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:32.697375 (+ 458us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:32.703062 (+ 5687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:32.703093 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:32.705785 (+ 2692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:32.708677 (+ 2892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:32.708680 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:32.708682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:32.708947 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:32.709685 (+ 738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:32.709689 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:32.709691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:32.709819 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:32.710874 (+ 1055us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:32.710877 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:32.711101 (+ 224us) client_negotiation.cc:769] Sending connection context
0812 11:12:32.711386 (+ 285us) client_negotiation.cc:240] Negotiation successful
0812 11:12:32.711927 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":387,"spinlock_wait_cycles":21248}
I20260812 11:12:32.713469 9096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:32.686783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:48343 (local address 127.17.44.195:46013)
0812 11:12:32.687646 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:32.687664 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:32.687714 (+ 50us) server_negotiation.cc:407] Connection header received
0812 11:12:32.688895 (+ 1181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:32.688905 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:32.689393 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:32.689808 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:32.691615 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:32.693590 (+ 1975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:32.695859 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:32.696721 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:32.696909 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:32.706232 (+ 9323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:32.706237 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:32.706247 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:32.706322 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:32.708507 (+ 2185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:32.709199 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:32.709209 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:32.709220 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:32.709450 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:32.710008 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:32.710011 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:32.710069 (+ 58us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:32.710670 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:32.710822 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:32.712661 (+ 1839us) server_negotiation.cc:299] Negotiation successful
0812 11:12:32.712988 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":161,"threads_started":1}
W20260812 11:12:32.715638 8503 leader_election.cc:341] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1ac60ba6fecc43bc856fae0dc226066e (127.17.44.195:46013): Not found: Tablet not found: 7603fe3770c94b1fb0190e4c3972888b
W20260812 11:12:32.715821 8506 leader_election.cc:341] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f8b256eb81ee4ceb93c293d70db13348 (127.17.44.194:35327): Not found: Tablet not found: 7603fe3770c94b1fb0190e4c3972888b
I20260812 11:12:32.715909 8506 leader_election.cc:302] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9bcc97a954b4d69888b38749ee663c9; no voters: 1ac60ba6fecc43bc856fae0dc226066e, f8b256eb81ee4ceb93c293d70db13348
I20260812 11:12:32.716148 9094 raft_consensus.cc:2726] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 11:12:32.737259 9099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:32.736836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.195:54897)
0812 11:12:32.737113 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:32.737166 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":103,"threads_started":1}
I20260812 11:12:33.391973 9100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:33.391453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.193:56399)
0812 11:12:33.391743 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:33.391818 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":96,"threads_started":1}
I20260812 11:12:33.531224 9101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:33.530631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.194:51755)
0812 11:12:33.531063 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:33.531128 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":115,"threads_started":1}
I20260812 11:12:33.738482 9102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:33.738046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.195:51737)
0812 11:12:33.738317 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:33.738370 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":82,"threads_started":1}
I20260812 11:12:34.393371 9103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:34.392888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.193:37831)
0812 11:12:34.393215 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:34.393272 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":84,"threads_started":1}
I20260812 11:12:34.465137 9104 raft_consensus.cc:482] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:12:34.465284 9104 raft_consensus.cc:504] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } }
I20260812 11:12:34.465655 9104 leader_election.cc:288] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ac60ba6fecc43bc856fae0dc226066e (127.17.44.195:46013), f8b256eb81ee4ceb93c293d70db13348 (127.17.44.194:35327)
W20260812 11:12:34.466921 8506 leader_election.cc:341] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f8b256eb81ee4ceb93c293d70db13348 (127.17.44.194:35327): Not found: Tablet not found: 7603fe3770c94b1fb0190e4c3972888b
W20260812 11:12:34.467049 8503 leader_election.cc:341] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1ac60ba6fecc43bc856fae0dc226066e (127.17.44.195:46013): Not found: Tablet not found: 7603fe3770c94b1fb0190e4c3972888b
I20260812 11:12:34.467127 8503 leader_election.cc:302] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9bcc97a954b4d69888b38749ee663c9; no voters: 1ac60ba6fecc43bc856fae0dc226066e, f8b256eb81ee4ceb93c293d70db13348
I20260812 11:12:34.467283 9104 raft_consensus.cc:2726] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 11:12:34.532629 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:34.532178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.194:49897)
0812 11:12:34.532458 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:34.532530 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":82,"threads_started":1}
I20260812 11:12:34.740137 9106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:34.739604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.195:45543)
0812 11:12:34.739967 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:34.740037 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":120,"threads_started":1}
I20260812 11:12:35.394838 9107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:35.394382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.193:55265)
0812 11:12:35.394678 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:35.394740 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":92,"threads_started":1}
I20260812 11:12:35.533941 9108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:35.533496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.194:44279)
0812 11:12:35.533786 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:35.533842 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":97,"threads_started":1}
I20260812 11:12:35.741632 9110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:35.741186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.195:43321)
0812 11:12:35.741481 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:35.741537 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":83,"threads_started":1}
I20260812 11:12:36.396463 9112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:36.395988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.193:56881)
0812 11:12:36.396305 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:36.396365 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":80,"threads_started":1}
I20260812 11:12:36.535434 9115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:36.534902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.194:34419)
0812 11:12:36.535270 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:36.535328 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":109,"threads_started":1}
I20260812 11:12:36.743438 9116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:36.742642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.195:34385)
0812 11:12:36.743196 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:36.743291 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":111,"threads_started":1}
I20260812 11:12:36.807092 9117 raft_consensus.cc:482] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:12:36.807260 9117 raft_consensus.cc:504] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } }
I20260812 11:12:36.807773 9117 leader_election.cc:288] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ac60ba6fecc43bc856fae0dc226066e (127.17.44.195:46013), f8b256eb81ee4ceb93c293d70db13348 (127.17.44.194:35327)
W20260812 11:12:36.808660 8503 leader_election.cc:341] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1ac60ba6fecc43bc856fae0dc226066e (127.17.44.195:46013): Not found: Tablet not found: 7603fe3770c94b1fb0190e4c3972888b
W20260812 11:12:36.808898 8506 leader_election.cc:341] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f8b256eb81ee4ceb93c293d70db13348 (127.17.44.194:35327): Not found: Tablet not found: 7603fe3770c94b1fb0190e4c3972888b
I20260812 11:12:36.808993 8506 leader_election.cc:302] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9bcc97a954b4d69888b38749ee663c9; no voters: 1ac60ba6fecc43bc856fae0dc226066e, f8b256eb81ee4ceb93c293d70db13348
I20260812 11:12:36.809243 9117 raft_consensus.cc:2726] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 11:12:37.397830 9118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:37.397365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.193:45467)
0812 11:12:37.397658 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:37.397716 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":99,"threads_started":1}
I20260812 11:12:37.536789 9119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:37.536316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.194:50349)
0812 11:12:37.536630 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:37.536685 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":100,"threads_started":1}
I20260812 11:12:37.745096 9121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:37.744283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.195:36863)
0812 11:12:37.744814 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:37.744920 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":98,"threads_started":1}
I20260812 11:12:38.399510 9122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:38.398834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.193:53015)
0812 11:12:38.399262 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:38.399365 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":137,"threads_started":1}
I20260812 11:12:38.470822 9124 raft_consensus.cc:482] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:12:38.471091 9124 raft_consensus.cc:504] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } }
I20260812 11:12:38.471702 9124 leader_election.cc:288] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ac60ba6fecc43bc856fae0dc226066e (127.17.44.195:46013), f8b256eb81ee4ceb93c293d70db13348 (127.17.44.194:35327)
W20260812 11:12:38.472901 8506 leader_election.cc:341] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f8b256eb81ee4ceb93c293d70db13348 (127.17.44.194:35327): Not found: Tablet not found: 7603fe3770c94b1fb0190e4c3972888b
W20260812 11:12:38.473209 8503 leader_election.cc:341] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1ac60ba6fecc43bc856fae0dc226066e (127.17.44.195:46013): Not found: Tablet not found: 7603fe3770c94b1fb0190e4c3972888b
I20260812 11:12:38.473282 8503 leader_election.cc:302] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9bcc97a954b4d69888b38749ee663c9; no voters: 1ac60ba6fecc43bc856fae0dc226066e, f8b256eb81ee4ceb93c293d70db13348
I20260812 11:12:38.473493 9124 raft_consensus.cc:2726] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 11:12:38.538085 9126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:38.537574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.194:55691)
0812 11:12:38.537904 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:38.537961 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":75,"threads_started":1}
I20260812 11:12:38.746696 9127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:38.745978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.195:33923)
0812 11:12:38.746517 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:38.746600 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":97,"threads_started":1}
I20260812 11:12:39.400897 9129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:39.400434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.193:54573)
0812 11:12:39.400730 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:39.400799 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":90,"threads_started":1}
I20260812 11:12:39.539831 9130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:39.539107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.194:51589)
0812 11:12:39.539609 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:39.539696 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":153,"threads_started":1}
I20260812 11:12:39.748107 9132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:39.747588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.195:37083)
0812 11:12:39.747926 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:39.747982 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":134,"threads_started":1}
I20260812 11:12:40.402187 9133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:40.401747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.193:53059)
0812 11:12:40.402033 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:40.402088 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":104,"threads_started":1}
I20260812 11:12:40.541651 9134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:40.540968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.194:52151)
0812 11:12:40.541406 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:40.541507 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":82,"threads_started":1}
I20260812 11:12:40.749572 9136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:40.749050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.195:32941)
0812 11:12:40.749413 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:40.749472 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":112,"threads_started":1}
I20260812 11:12:40.988389 9140 raft_consensus.cc:482] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:12:40.988665 9140 raft_consensus.cc:504] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ac60ba6fecc43bc856fae0dc226066e" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46013 } } peers { permanent_uuid: "f8b256eb81ee4ceb93c293d70db13348" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 35327 } } peers { permanent_uuid: "e9bcc97a954b4d69888b38749ee663c9" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 45743 } }
I20260812 11:12:40.989403 9140 leader_election.cc:288] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ac60ba6fecc43bc856fae0dc226066e (127.17.44.195:46013), f8b256eb81ee4ceb93c293d70db13348 (127.17.44.194:35327)
W20260812 11:12:40.990443 8503 leader_election.cc:341] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1ac60ba6fecc43bc856fae0dc226066e (127.17.44.195:46013): Not found: Tablet not found: 7603fe3770c94b1fb0190e4c3972888b
W20260812 11:12:40.990621 8506 leader_election.cc:341] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f8b256eb81ee4ceb93c293d70db13348 (127.17.44.194:35327): Not found: Tablet not found: 7603fe3770c94b1fb0190e4c3972888b
I20260812 11:12:40.990700 8506 leader_election.cc:302] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9bcc97a954b4d69888b38749ee663c9; no voters: 1ac60ba6fecc43bc856fae0dc226066e, f8b256eb81ee4ceb93c293d70db13348
I20260812 11:12:40.990877 9140 raft_consensus.cc:2726] T 7603fe3770c94b1fb0190e4c3972888b P e9bcc97a954b4d69888b38749ee663c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 11:12:41.403931 9141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:41.403324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.193:56771)
0812 11:12:41.403716 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:41.403811 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":95,"threads_started":1}
I20260812 11:12:41.543327 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:41.542719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.194:37593)
0812 11:12:41.543165 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:41.543224 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":103,"threads_started":1}
I20260812 11:12:41.750905 9145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:41.750474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.195:42295)
0812 11:12:41.750755 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:41.750811 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":84,"threads_started":1}
I20260812 11:12:42.405759 9146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:42.405035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.193:57275)
0812 11:12:42.405476 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:42.405645 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":108,"threads_started":1}
I20260812 11:12:42.544701 9148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:42.544248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.194:52513)
0812 11:12:42.544542 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:42.544601 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":82,"threads_started":1}
I20260812 11:12:42.753110 9149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:42.752413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.195:52287)
0812 11:12:42.752881 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:42.752967 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":117,"threads_started":1}
I20260812 11:12:43.407040 9150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:43.406546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.193:45727)
0812 11:12:43.406813 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:43.406869 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":79,"threads_started":1}
I20260812 11:12:43.546144 9151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:43.545627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.194:41399)
0812 11:12:43.545986 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:43.546044 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":189,"threads_started":1}
I20260812 11:12:43.755190 9152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:43.754474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.195:42931)
0812 11:12:43.754894 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:43.755033 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":93,"threads_started":1}
I20260812 11:12:44.408390 9154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:44.407910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.193:35573)
0812 11:12:44.408219 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:44.408287 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":93,"threads_started":1}
I20260812 11:12:44.547514 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:44.547022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.194:38009)
0812 11:12:44.547361 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:44.547416 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":82,"threads_started":1}
I20260812 11:12:44.756654 9156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:44.756203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.195:49307)
0812 11:12:44.756488 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:44.756543 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":78,"threads_started":1}
I20260812 11:12:45.409917 9157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:45.409499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.193:35063)
0812 11:12:45.409765 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:45.409819 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":87,"threads_started":1}
I20260812 11:12:45.549055 9158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:45.548468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.194:40601)
0812 11:12:45.548822 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:45.548913 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":88,"threads_started":1}
I20260812 11:12:45.758061 9160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:45.757599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.195:37261)
0812 11:12:45.757906 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:45.757965 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":84,"threads_started":1}
I20260812 11:12:46.412063 9161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:46.411494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.193:37327)
0812 11:12:46.411876 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:46.411953 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":104,"threads_started":1}
I20260812 11:12:46.550913 9162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:46.550335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.194:56389)
0812 11:12:46.550616 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:46.550679 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":78,"threads_started":1}
I20260812 11:12:46.750767 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 8424
I20260812 11:12:46.760388 9163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:46.759384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:36829 (local address 127.17.44.195:32813)
0812 11:12:46.760180 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:46.760256 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":314,"threads_started":1}
I20260812 11:12:46.767732 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 8682
I20260812 11:12:46.788575 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 8887
2026-08-12T11:12:46Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19900 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:12:46 dist-test-slave-drl0 krb5kdc[9169](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:12:46 dist-test-slave-drl0 krb5kdc[9169](info): set up 2 sockets
Aug 12 11:12:46 dist-test-slave-drl0 krb5kdc[9169](info): commencing operation
krb5kdc: starting...
W20260812 11:12:48.124491 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:12:48 dist-test-slave-drl0 krb5kdc[9169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:12:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:12:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:12:48.264587 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:42965
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:42965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:12:48.381783 9187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:48.382225 9187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:48.382318 9187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:48.387761 9187 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 11:12:48.387897 9187 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 11:12:48.387928 9187 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 11:12:48.387954 9187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:12:48.388001 9187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:48.388068 9187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:48.388108 9187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:12:48.388140 9187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:12:48.393587 9187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:42965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:42965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:48.394663 9187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:48.395725 9187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:48.406009 9194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:48.406052 9196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:48.406103 9187 server_base.cc:1034] running on GCE node
W20260812 11:12:48.406052 9193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:48.407186 9187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:48.408278 9187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:48.409503 9187 hybrid_clock.cc:648] HybridClock initialized: now 1786533168409485 us; error 31 us; skew 500 ppm
Aug 12 11:12:48 dist-test-slave-drl0 krb5kdc[9169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533168, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:48.412545 9187 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:12:48.413605 9187 webserver.cc:460] Webserver started at http://127.17.44.254:38247/ using document root <none> and password file <none>
I20260812 11:12:48.414206 9187 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:48.414306 9187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:12:48.414565 9187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:12:48.416790 9187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "e11a4e54ab5e486bbc1162221cdcc66e"
format_stamp: "Formatted at 2026-08-12 11:12:48 on dist-test-slave-drl0"
server_key: "d9537b1b73f212dab36936e68f774fb8"
server_key_iv: "f2e95ccf4af2aaa2a51d4dddb7bbe705"
server_key_version: "encryptionkey@0"
I20260812 11:12:48.417368 9187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "e11a4e54ab5e486bbc1162221cdcc66e"
format_stamp: "Formatted at 2026-08-12 11:12:48 on dist-test-slave-drl0"
server_key: "d9537b1b73f212dab36936e68f774fb8"
server_key_iv: "f2e95ccf4af2aaa2a51d4dddb7bbe705"
server_key_version: "encryptionkey@0"
I20260812 11:12:48.421164 9187 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 11:12:48.423789 9204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:48.425283 9187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 11:12:48.425458 9187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "e11a4e54ab5e486bbc1162221cdcc66e"
format_stamp: "Formatted at 2026-08-12 11:12:48 on dist-test-slave-drl0"
server_key: "d9537b1b73f212dab36936e68f774fb8"
server_key_iv: "f2e95ccf4af2aaa2a51d4dddb7bbe705"
server_key_version: "encryptionkey@0"
I20260812 11:12:48.425582 9187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:48.441211 9187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:48.445379 9187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:48.445637 9187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:48.455571 9187 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:42965
I20260812 11:12:48.455873 9280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:42965 every 8 connection(s)
I20260812 11:12:48.456722 9187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:12:48.459805 9281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:12:48.462769 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 9187
I20260812 11:12:48.462903 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:12:48.463238 17587 external_mini_cluster.cc:1428] Setting key f379513159d838f099431ccca55d6592
I20260812 11:12:48.466425 9281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e: Bootstrap starting.
I20260812 11:12:48.468819 9281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e: Neither blocks nor log segments found. Creating new log.
I20260812 11:12:48.469671 9281 log.cc:824] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e: Log is configured to *not* fsync() on all Append() calls
I20260812 11:12:48.471747 9281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e: No bootstrap required, opened a new log
Aug 12 11:12:48 dist-test-slave-drl0 krb5kdc[9169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:12:48.476464 9281 raft_consensus.cc:348] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11a4e54ab5e486bbc1162221cdcc66e" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42965 } }
I20260812 11:12:48.476776 9281 raft_consensus.cc:374] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:12:48.476850 9281 raft_consensus.cc:729] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e11a4e54ab5e486bbc1162221cdcc66e, State: Initialized, Role: FOLLOWER
I20260812 11:12:48.477385 9281 consensus_queue.cc:260] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11a4e54ab5e486bbc1162221cdcc66e" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42965 } }
I20260812 11:12:48.477548 9281 raft_consensus.cc:388] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:12:48.477622 9281 raft_consensus.cc:482] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:12:48.477769 9281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:12:48.479050 9281 raft_consensus.cc:504] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11a4e54ab5e486bbc1162221cdcc66e" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42965 } }
I20260812 11:12:48.479424 9281 leader_election.cc:302] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e11a4e54ab5e486bbc1162221cdcc66e; no voters:
I20260812 11:12:48.479579 9286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:48.465088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40926 (local address 127.17.44.254:42965)
0812 11:12:48.465740 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:48.465757 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:48.465797 (+ 40us) server_negotiation.cc:407] Connection header received
0812 11:12:48.466452 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:48.466469 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:48.466867 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:48.467297 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:48.468174 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:48.469291 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:48.470194 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:48.470667 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:48.471032 (+ 365us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:48.473856 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:48.473863 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:48.473880 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:48.473912 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:48.476088 (+ 2176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:48.476557 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:48.476563 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:48.476569 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:48.476704 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:48.477081 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:48.477088 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:48.477092 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:48.477601 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:48.477814 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:48.478178 (+ 364us) server_negotiation.cc:299] Negotiation successful
0812 11:12:48.478483 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":300,"threads_started":1}
I20260812 11:12:48.479760 9281 leader_election.cc:288] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:12:48.479879 9289 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:12:48.480105 9289 raft_consensus.cc:686] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 1 LEADER]: Becoming Leader. State: Replica: e11a4e54ab5e486bbc1162221cdcc66e, State: Running, Role: LEADER
I20260812 11:12:48.480475 9289 consensus_queue.cc:237] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11a4e54ab5e486bbc1162221cdcc66e" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42965 } }
I20260812 11:12:48.481086 9281 sys_catalog.cc:564] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:12:48.481842 9292 sys_catalog.cc:455] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e11a4e54ab5e486bbc1162221cdcc66e. Latest consensus state: current_term: 1 leader_uuid: "e11a4e54ab5e486bbc1162221cdcc66e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11a4e54ab5e486bbc1162221cdcc66e" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42965 } } }
I20260812 11:12:48.481981 9292 sys_catalog.cc:458] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [sys.catalog]: This master's current role is: LEADER
I20260812 11:12:48.482242 9291 sys_catalog.cc:455] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e11a4e54ab5e486bbc1162221cdcc66e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11a4e54ab5e486bbc1162221cdcc66e" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42965 } } }
I20260812 11:12:48.482378 9291 sys_catalog.cc:458] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [sys.catalog]: This master's current role is: LEADER
I20260812 11:12:48.483219 9300 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:12:48.485831 9300 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:12:48.490979 9300 catalog_manager.cc:1347] Generated new cluster ID: 40fef90bb8a04147982b4aeade7fda17
I20260812 11:12:48.491036 9300 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:12:48.504253 9300 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:12:48.505445 9300 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:12:48.516203 9300 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e: Generated new TSK 0
I20260812 11:12:48.516976 9300 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
I20260812 11:12:48.553606 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:42965
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:12:48.675740 9322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:48.676059 9322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:48.676138 9322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:48.680301 9322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:48.680392 9322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:48.680480 9322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:12:48.686623 9322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:42965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:48.687772 9322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:48.688853 9322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:48.698853 9335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:48.698812 9330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:48.698813 9333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:48.700040 9322 server_base.cc:1034] running on GCE node
I20260812 11:12:48.700716 9322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:48.701560 9322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:48.702808 9322 hybrid_clock.cc:648] HybridClock initialized: now 1786533168702778 us; error 40 us; skew 500 ppm
Aug 12 11:12:48 dist-test-slave-drl0 krb5kdc[9169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533168, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:48.706014 9322 init.cc:377] Logged in from keytab as kudu/127.17.44.193@KRBTEST.COM (short username kudu)
I20260812 11:12:48.707021 9322 webserver.cc:460] Webserver started at http://127.17.44.193:40445/ using document root <none> and password file <none>
I20260812 11:12:48.707612 9322 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:48.707703 9322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:12:48.707940 9322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:12:48.710072 9322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "3cb5d354b65e44a2b85e18650db4d504"
format_stamp: "Formatted at 2026-08-12 11:12:48 on dist-test-slave-drl0"
server_key: "62d8732f984fd8728b64073943ec4d84"
server_key_iv: "bbee7c5fb387d3f02c1d2da362f1d466"
server_key_version: "encryptionkey@0"
I20260812 11:12:48.710657 9322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "3cb5d354b65e44a2b85e18650db4d504"
format_stamp: "Formatted at 2026-08-12 11:12:48 on dist-test-slave-drl0"
server_key: "62d8732f984fd8728b64073943ec4d84"
server_key_iv: "bbee7c5fb387d3f02c1d2da362f1d466"
server_key_version: "encryptionkey@0"
I20260812 11:12:48.715027 9322 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 11:12:48.717595 9346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:48.719185 9322 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 11:12:48.719327 9322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "3cb5d354b65e44a2b85e18650db4d504"
format_stamp: "Formatted at 2026-08-12 11:12:48 on dist-test-slave-drl0"
server_key: "62d8732f984fd8728b64073943ec4d84"
server_key_iv: "bbee7c5fb387d3f02c1d2da362f1d466"
server_key_version: "encryptionkey@0"
I20260812 11:12:48.719452 9322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:48.747207 9322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:48.751178 9322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:48.751402 9322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:48.752051 9322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:12:48.752924 9322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:12:48.752996 9322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:48.753068 9322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:12:48.753119 9322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:48.763566 9322 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:46467
I20260812 11:12:48.763731 9518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:46467 every 8 connection(s)
I20260812 11:12:48.764549 9322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:12:48.772231 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 9322
I20260812 11:12:48.772341 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:12:48.772666 17587 external_mini_cluster.cc:1428] Setting key 48f25905b265f258a14e2d1369c667ae
Aug 12 11:12:48 dist-test-slave-drl0 krb5kdc[9169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533168, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:12:48.780086 9286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:48.767312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:55787 (local address 127.17.44.254:42965)
0812 11:12:48.767519 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:48.767523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:48.768468 (+ 945us) server_negotiation.cc:407] Connection header received
0812 11:12:48.769889 (+ 1421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:48.769892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:48.769954 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:48.770062 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:48.771265 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:48.771806 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:48.773278 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:48.773570 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:48.773675 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:48.776419 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:48.776428 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:48.776430 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:48.776453 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:48.777913 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:48.778460 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:48.778463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:48.778465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:48.778542 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:48.778909 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:48.778916 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:48.778918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:48.779141 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:48.779262 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:48.779773 (+ 511us) server_negotiation.cc:299] Negotiation successful
0812 11:12:48.779912 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260812 11:12:48.780931 9524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:48.767754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.193:55787)
0812 11:12:48.768278 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:48.768323 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:48.769651 (+ 1328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:48.770239 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:48.770245 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:48.770600 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:48.771105 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:48.771119 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:48.771966 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:48.771970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:48.773090 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:48.773102 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:48.773747 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:48.773752 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:48.773958 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:48.774679 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:48.774696 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:48.776239 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:48.778042 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:48.778048 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:48.778062 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:48.778328 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:48.778638 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:48.778642 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:48.778644 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:48.778791 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:48.779261 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:48.779269 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:48.779525 (+ 256us) client_negotiation.cc:769] Sending connection context
0812 11:12:48.779715 (+ 190us) client_negotiation.cc:240] Negotiation successful
0812 11:12:48.780043 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":131,"threads_started":1}
I20260812 11:12:48.782447 9520 heartbeater.cc:347] Connected to a master server at 127.17.44.254:42965
I20260812 11:12:48.782739 9520 heartbeater.cc:464] Registering TS with master...
I20260812 11:12:48.783317 9520 heartbeater.cc:511] Master 127.17.44.254:42965 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.44.194@KRBTEST.COM; defaulting to no policy
I20260812 11:12:48.785341 9225 ts_manager.cc:194] Registered new tserver with Master: 3cb5d354b65e44a2b85e18650db4d504 (127.17.44.193:46467)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.194@KRBTEST.COM" created.
I20260812 11:12:48.786625 9225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.193@KRBTEST.COM'} at 127.17.44.193:55787
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
I20260812 11:12:48.806023 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:42965
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:12:48.928188 9527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:48.928570 9527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:48.928664 9527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:48.935555 9527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:48.935634 9527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:48.935730 9527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:12:48.942457 9527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:42965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:48.943641 9527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:48.944655 9527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:48.954741 9547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:48.954761 9544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:48.954741 9537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:48.954900 9527 server_base.cc:1034] running on GCE node
I20260812 11:12:48.955458 9527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:48.956216 9527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:48.957406 9527 hybrid_clock.cc:648] HybridClock initialized: now 1786533168957363 us; error 56 us; skew 500 ppm
Aug 12 11:12:48 dist-test-slave-drl0 krb5kdc[9169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533168, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:48.961462 9527 init.cc:377] Logged in from keytab as kudu/127.17.44.194@KRBTEST.COM (short username kudu)
I20260812 11:12:48.962383 9527 webserver.cc:460] Webserver started at http://127.17.44.194:46293/ using document root <none> and password file <none>
I20260812 11:12:48.963115 9527 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:48.963210 9527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:12:48.963454 9527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:12:48.965520 9527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "bb17c21d0ed14f238506c985e17099da"
format_stamp: "Formatted at 2026-08-12 11:12:48 on dist-test-slave-drl0"
server_key: "9827b9bd2d0cfec07cff7cf005a35042"
server_key_iv: "c4e7559908d10abef0b580a987ccd072"
server_key_version: "encryptionkey@0"
I20260812 11:12:48.966094 9527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "bb17c21d0ed14f238506c985e17099da"
format_stamp: "Formatted at 2026-08-12 11:12:48 on dist-test-slave-drl0"
server_key: "9827b9bd2d0cfec07cff7cf005a35042"
server_key_iv: "c4e7559908d10abef0b580a987ccd072"
server_key_version: "encryptionkey@0"
I20260812 11:12:48.970337 9527 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 11:12:48.973227 9558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:48.974663 9527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:12:48.974817 9527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "bb17c21d0ed14f238506c985e17099da"
format_stamp: "Formatted at 2026-08-12 11:12:48 on dist-test-slave-drl0"
server_key: "9827b9bd2d0cfec07cff7cf005a35042"
server_key_iv: "c4e7559908d10abef0b580a987ccd072"
server_key_version: "encryptionkey@0"
I20260812 11:12:48.974962 9527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:49.002190 9527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:49.006284 9527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:49.006522 9527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:49.007236 9527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:12:49.008204 9527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:12:49.008298 9527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:49.008399 9527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:12:49.008517 9527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:49.018451 9527 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:34241
I20260812 11:12:49.018477 9727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:34241 every 8 connection(s)
I20260812 11:12:49.019436 9527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:12:49.024518 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 9527
I20260812 11:12:49.024618 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:12:49.025012 17587 external_mini_cluster.cc:1428] Setting key b20d93970726d4ea56d556da2f897a68
Aug 12 11:12:49 dist-test-slave-drl0 krb5kdc[9169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533168, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.44.195@KRBTEST.COM; defaulting to no policy
I20260812 11:12:49.036945 9286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:49.022246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:48977 (local address 127.17.44.254:42965)
0812 11:12:49.022386 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:49.022390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:49.023780 (+ 1390us) server_negotiation.cc:407] Connection header received
0812 11:12:49.025535 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:49.025538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:49.025589 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:49.025669 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:49.027316 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.028138 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.029479 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.029709 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.029838 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:49.032831 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:49.032834 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:49.032836 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:49.032860 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:49.034588 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:49.035214 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:49.035217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:49.035219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:49.035292 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:49.035643 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:49.035647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:49.035649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:49.035848 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:49.035962 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:49.036695 (+ 733us) server_negotiation.cc:299] Negotiation successful
0812 11:12:49.036814 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 11:12:49.038316 9730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:49.022564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.194:48977)
0812 11:12:49.023545 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:49.023601 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:49.025271 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:49.025877 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:49.025905 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:49.026413 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:49.027151 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:49.027167 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.028337 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.028340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:49.029301 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:49.029311 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.029880 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.029884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:49.030078 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:49.030716 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:49.030732 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:49.032649 (+ 1917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:49.034718 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:49.034724 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:49.034741 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:49.035096 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:49.035386 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:49.035390 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:49.035392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:49.035538 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:49.035963 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:49.035970 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:49.036242 (+ 272us) client_negotiation.cc:769] Sending connection context
0812 11:12:49.037228 (+ 986us) client_negotiation.cc:240] Negotiation successful
0812 11:12:49.037516 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":772,"thread_start_us":556,"threads_started":1}
I20260812 11:12:49.039657 9728 heartbeater.cc:347] Connected to a master server at 127.17.44.254:42965
I20260812 11:12:49.039973 9728 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.195@KRBTEST.COM" created.
I20260812 11:12:49.040573 9728 heartbeater.cc:511] Master 127.17.44.254:42965 requested a full tablet report, sending...
I20260812 11:12:49.042333 9225 ts_manager.cc:194] Registered new tserver with Master: bb17c21d0ed14f238506c985e17099da (127.17.44.194:34241)
I20260812 11:12:49.043252 9225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.194@KRBTEST.COM'} at 127.17.44.194:48977
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
I20260812 11:12:49.059398 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:42965
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:12:49.191933 9734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:12:49.192485 9734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:12:49.192545 9734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:12:49.197783 9734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:12:49.197860 9734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:12:49.197925 9734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:12:49.204211 9734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:42965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:12:49.205832 9734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:12:49.207428 9734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:12:49.218384 9750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:49.218421 9745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:12:49.218559 9746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:12:49.218593 9734 server_base.cc:1034] running on GCE node
I20260812 11:12:49.219211 9734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:12:49.219784 9734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:12:49.221101 9734 hybrid_clock.cc:648] HybridClock initialized: now 1786533169221082 us; error 109 us; skew 500 ppm
Aug 12 11:12:49 dist-test-slave-drl0 krb5kdc[9169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533169, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:12:49.224942 9734 init.cc:377] Logged in from keytab as kudu/127.17.44.195@KRBTEST.COM (short username kudu)
I20260812 11:12:49.226111 9734 webserver.cc:460] Webserver started at http://127.17.44.195:34383/ using document root <none> and password file <none>
I20260812 11:12:49.226805 9734 fs_manager.cc:359] Metadata directory not provided
I20260812 11:12:49.226874 9734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:12:49.227193 9734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:12:49.229866 9734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "acba0798196d48d691a96b28eaed9b11"
format_stamp: "Formatted at 2026-08-12 11:12:49 on dist-test-slave-drl0"
server_key: "f7883bfd4ed9853bab4033de0aaeca3b"
server_key_iv: "498fb7c6b4df3389e041801c76ac2a35"
server_key_version: "encryptionkey@0"
I20260812 11:12:49.230535 9734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "acba0798196d48d691a96b28eaed9b11"
format_stamp: "Formatted at 2026-08-12 11:12:49 on dist-test-slave-drl0"
server_key: "f7883bfd4ed9853bab4033de0aaeca3b"
server_key_iv: "498fb7c6b4df3389e041801c76ac2a35"
server_key_version: "encryptionkey@0"
I20260812 11:12:49.234562 9734 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 11:12:49.237730 9761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:49.239602 9734 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 11:12:49.239724 9734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "acba0798196d48d691a96b28eaed9b11"
format_stamp: "Formatted at 2026-08-12 11:12:49 on dist-test-slave-drl0"
server_key: "f7883bfd4ed9853bab4033de0aaeca3b"
server_key_iv: "498fb7c6b4df3389e041801c76ac2a35"
server_key_version: "encryptionkey@0"
I20260812 11:12:49.239921 9734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:12:49.261528 9734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:12:49.264899 9734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:12:49.265064 9734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:12:49.265601 9734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:12:49.266458 9734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:12:49.266508 9734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:49.266547 9734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:12:49.266562 9734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:12:49.278028 9734 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:43595
I20260812 11:12:49.278069 9926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:43595 every 8 connection(s)
I20260812 11:12:49.279136 9734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:12:49.280370 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 9734
I20260812 11:12:49.280519 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:12:49.280879 17587 external_mini_cluster.cc:1428] Setting key dda211d764f3af11816a19f42084e011
Aug 12 11:12:49 dist-test-slave-drl0 krb5kdc[9169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533169, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:12:49.294793 9286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:49.281168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:56891 (local address 127.17.44.254:42965)
0812 11:12:49.281332 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:49.281337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:49.282388 (+ 1051us) server_negotiation.cc:407] Connection header received
0812 11:12:49.284128 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:49.284131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:49.284176 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:49.284251 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:49.286051 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.286631 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.288050 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.288285 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.288378 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:49.291031 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:49.291034 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:49.291036 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:49.291060 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:49.292437 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:49.293019 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:49.293022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:49.293023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:49.293104 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:49.293499 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:49.293502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:49.293503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:49.293684 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:49.293910 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:49.294494 (+ 584us) server_negotiation.cc:299] Negotiation successful
0812 11:12:49.294616 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 11:12:49.295764 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:49.281664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.195:56891)
0812 11:12:49.282203 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:49.282281 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:49.283892 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:49.284624 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:49.284635 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:49.285182 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:12:49.285843 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:49.285861 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.286848 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.286853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:49.287881 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:49.287893 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.288446 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.288449 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:49.288598 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:49.289203 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:12:49.289219 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:12:49.290805 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:12:49.292629 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:49.292634 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:49.292646 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:49.292894 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:49.293236 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:12:49.293239 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:12:49.293240 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:12:49.293385 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:12:49.293910 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:12:49.293918 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:12:49.294249 (+ 331us) client_negotiation.cc:769] Sending connection context
0812 11:12:49.294451 (+ 202us) client_negotiation.cc:240] Negotiation successful
0812 11:12:49.294782 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":138,"threads_started":1}
I20260812 11:12:49.297791 9927 heartbeater.cc:347] Connected to a master server at 127.17.44.254:42965
I20260812 11:12:49.298074 9927 heartbeater.cc:464] Registering TS with master...
I20260812 11:12:49.298596 9927 heartbeater.cc:511] Master 127.17.44.254:42965 requested a full tablet report, sending...
I20260812 11:12:49.300293 9225 ts_manager.cc:194] Registered new tserver with Master: acba0798196d48d691a96b28eaed9b11 (127.17.44.195:43595)
I20260812 11:12:49.300879 9225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.195@KRBTEST.COM'} at 127.17.44.195:56891
I20260812 11:12:49.307345 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:12:49.318126 9286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:49.309533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40930 (local address 127.17.44.254:42965)
0812 11:12:49.309843 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:49.309849 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:49.310023 (+ 174us) server_negotiation.cc:407] Connection header received
0812 11:12:49.310245 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:49.310253 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:49.310303 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:49.310406 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:12:49.311331 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.312012 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.312949 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.313266 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.313412 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:49.314988 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:12:49.314991 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:12:49.314993 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:12:49.315017 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:12:49.316363 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:49.316815 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:49.316833 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:49.316834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:49.316913 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:12:49.317213 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:12:49.317220 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:12:49.317222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:12:49.317444 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:12:49.317584 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:49.317836 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 11:12:49.317966 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173}
I20260812 11:12:49.321208 9225 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40930:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:12:49.324827 9225 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:12:49.341131 9959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:49.335729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48838 (local address 127.17.44.193:46467)
0812 11:12:49.336135 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:49.336141 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:49.336159 (+ 18us) server_negotiation.cc:407] Connection header received
0812 11:12:49.336228 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:49.336232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:49.336394 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:49.336521 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:12:49.336887 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.338048 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.339860 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.340435 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.340566 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:49.340728 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:49.340864 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 11:12:49.340978 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":127,"threads_started":1}
I20260812 11:12:49.341118 9955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:49.335189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:46467 (local address 127.0.0.1:48838)
0812 11:12:49.335458 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:49.335473 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:49.335567 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:49.336599 (+ 1032us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:49.336601 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:49.336623 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:12:49.336731 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:49.336735 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.338215 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.338219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:49.339735 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:49.339745 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.340567 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.340571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:49.340675 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:49.340714 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 11:12:49.340824 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 11:12:49.340967 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":59,"thread_start_us":105,"threads_started":1}
I20260812 11:12:49.341987 9951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:49.334560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47548 (local address 127.17.44.195:43595)
0812 11:12:49.335031 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:49.335054 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:49.335662 (+ 608us) server_negotiation.cc:407] Connection header received
0812 11:12:49.335913 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:49.335921 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:49.336122 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:49.336229 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:12:49.336850 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.338045 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.339543 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.340430 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.340708 (+ 278us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:49.341008 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:49.341733 (+ 725us) server_negotiation.cc:299] Negotiation successful
0812 11:12:49.341840 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":111,"threads_started":1}
I20260812 11:12:49.342396 9952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:49.334353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:43595 (local address 127.0.0.1:47548)
0812 11:12:49.335572 (+ 1219us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:49.335585 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:49.335758 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:49.336529 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:49.336532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:49.336544 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:12:49.336712 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:49.336720 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.338191 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.338194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:49.339411 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:49.339422 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.341469 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.341473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:49.341563 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:49.341587 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 11:12:49.342198 (+ 611us) client_negotiation.cc:240] Negotiation successful
0812 11:12:49.342275 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"mutex_wait_us":68,"thread_start_us":126,"threads_started":1}
I20260812 11:12:49.342783 9949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:49.334752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:34241 (local address 127.0.0.1:38430)
0812 11:12:49.335212 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:49.335267 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:49.335398 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:49.336451 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:49.336461 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:49.336492 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:12:49.336712 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:49.336720 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.339654 (+ 2934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.339658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:49.340886 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:49.340911 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.342276 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.342280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:49.342378 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:49.342411 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 11:12:49.342465 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 11:12:49.342549 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":32,"thread_start_us":108,"threads_started":1}
I20260812 11:12:49.343719 9958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:49.335258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38430 (local address 127.17.44.194:34241)
0812 11:12:49.335932 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:49.335944 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:49.335966 (+ 22us) server_negotiation.cc:407] Connection header received
0812 11:12:49.336045 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:49.336050 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:49.336238 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:49.336395 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:12:49.338215 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.339477 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.341312 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.342117 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.343163 (+ 1046us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:49.343338 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:49.343455 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 11:12:49.343549 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":161,"threads_started":1}
I20260812 11:12:49.344488 9418 tablet_service.cc:1467] Processing CreateTablet for tablet 81ace18129de47c0a994f1f1d208d0cf (DEFAULT_TABLE table=test-table [id=7eb3bd3f564a4cc4be90c348125de32c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:12:49.345037 9828 tablet_service.cc:1467] Processing CreateTablet for tablet 81ace18129de47c0a994f1f1d208d0cf (DEFAULT_TABLE table=test-table [id=7eb3bd3f564a4cc4be90c348125de32c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:12:49.345680 9418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81ace18129de47c0a994f1f1d208d0cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:12:49.345990 9828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81ace18129de47c0a994f1f1d208d0cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:12:49.346194 9633 tablet_service.cc:1467] Processing CreateTablet for tablet 81ace18129de47c0a994f1f1d208d0cf (DEFAULT_TABLE table=test-table [id=7eb3bd3f564a4cc4be90c348125de32c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:12:49.347186 9633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81ace18129de47c0a994f1f1d208d0cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:12:49.353114 9963 tablet_bootstrap.cc:492] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11: Bootstrap starting.
I20260812 11:12:49.353554 9961 tablet_bootstrap.cc:492] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504: Bootstrap starting.
I20260812 11:12:49.353905 9964 tablet_bootstrap.cc:492] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da: Bootstrap starting.
I20260812 11:12:49.356048 9963 tablet_bootstrap.cc:654] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11: Neither blocks nor log segments found. Creating new log.
I20260812 11:12:49.356361 9964 tablet_bootstrap.cc:654] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da: Neither blocks nor log segments found. Creating new log.
I20260812 11:12:49.357035 9961 tablet_bootstrap.cc:654] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504: Neither blocks nor log segments found. Creating new log.
I20260812 11:12:49.357260 9964 log.cc:824] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da: Log is configured to *not* fsync() on all Append() calls
I20260812 11:12:49.357589 9963 log.cc:824] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11: Log is configured to *not* fsync() on all Append() calls
I20260812 11:12:49.357806 9961 log.cc:824] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504: Log is configured to *not* fsync() on all Append() calls
I20260812 11:12:49.359345 9964 tablet_bootstrap.cc:492] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da: No bootstrap required, opened a new log
I20260812 11:12:49.359561 9964 ts_tablet_manager.cc:1397] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:12:49.359599 9961 tablet_bootstrap.cc:492] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504: No bootstrap required, opened a new log
I20260812 11:12:49.359599 9963 tablet_bootstrap.cc:492] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11: No bootstrap required, opened a new log
I20260812 11:12:49.359782 9963 ts_tablet_manager.cc:1397] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20260812 11:12:49.359782 9961 ts_tablet_manager.cc:1397] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 11:12:49.362860 9964 raft_consensus.cc:348] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } } peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } }
I20260812 11:12:49.363219 9964 raft_consensus.cc:374] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:12:49.363279 9964 raft_consensus.cc:729] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb17c21d0ed14f238506c985e17099da, State: Initialized, Role: FOLLOWER
I20260812 11:12:49.363830 9964 consensus_queue.cc:260] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } } peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } }
I20260812 11:12:49.364508 9728 heartbeater.cc:503] Master 127.17.44.254:42965 was elected leader, sending a full tablet report...
I20260812 11:12:49.364624 9963 raft_consensus.cc:348] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } } peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } }
I20260812 11:12:49.364879 9964 ts_tablet_manager.cc:1428] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 11:12:49.364892 9963 raft_consensus.cc:374] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:12:49.364952 9963 raft_consensus.cc:729] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acba0798196d48d691a96b28eaed9b11, State: Initialized, Role: FOLLOWER
I20260812 11:12:49.364833 9961 raft_consensus.cc:348] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } } peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } }
I20260812 11:12:49.365056 9961 raft_consensus.cc:374] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:12:49.365098 9961 raft_consensus.cc:729] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cb5d354b65e44a2b85e18650db4d504, State: Initialized, Role: FOLLOWER
I20260812 11:12:49.365391 9963 consensus_queue.cc:260] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } } peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } }
I20260812 11:12:49.365442 9961 consensus_queue.cc:260] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } } peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } }
I20260812 11:12:49.366325 9963 ts_tablet_manager.cc:1428] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260812 11:12:49.366325 9961 ts_tablet_manager.cc:1428] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 11:12:49.366709 9520 heartbeater.cc:503] Master 127.17.44.254:42965 was elected leader, sending a full tablet report...
I20260812 11:12:49.367023 9927 heartbeater.cc:503] Master 127.17.44.254:42965 was elected leader, sending a full tablet report...
W20260812 11:12:49.516053 9523 tablet.cc:2369] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:12:49.520681 9729 tablet.cc:2369] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:12:49.530848 9930 tablet.cc:2369] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:12:49.710668 9970 raft_consensus.cc:482] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:12:49.711079 9970 raft_consensus.cc:504] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } } peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } }
I20260812 11:12:49.712288 9970 leader_election.cc:288] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3cb5d354b65e44a2b85e18650db4d504 (127.17.44.193:46467), bb17c21d0ed14f238506c985e17099da (127.17.44.194:34241)
I20260812 11:12:49.715425 9972 raft_consensus.cc:482] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:12:49.715610 9972 raft_consensus.cc:504] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } } peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } }
I20260812 11:12:49.716909 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:49.712510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:46467 (local address 127.17.44.195:42495)
0812 11:12:49.712638 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:49.712654 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:49.712752 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:49.713166 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:49.713169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:49.713192 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:12:49.713502 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:49.713516 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.714422 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.714425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:49.715587 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:49.715598 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.716433 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.716437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:49.716546 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:49.716575 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 11:12:49.716659 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 11:12:49.716741 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":17,"spinlock_wait_cycles":47360}
I20260812 11:12:49.716941 9959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:49.712628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:42495 (local address 127.17.44.193:46467)
0812 11:12:49.712783 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:49.712791 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:49.712809 (+ 18us) server_negotiation.cc:407] Connection header received
0812 11:12:49.712886 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:49.712892 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:49.713027 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:49.713141 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:12:49.713678 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.714287 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.715709 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.716303 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.716434 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:49.716489 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:49.716691 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 11:12:49.716777 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 11:12:49.717494 9972 leader_election.cc:288] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acba0798196d48d691a96b28eaed9b11 (127.17.44.195:43595), bb17c21d0ed14f238506c985e17099da (127.17.44.194:34241)
I20260812 11:12:49.717607 9958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:49.712807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:54309 (local address 127.17.44.194:34241)
0812 11:12:49.712932 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:49.712938 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:49.713182 (+ 244us) server_negotiation.cc:407] Connection header received
0812 11:12:49.713377 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:49.713392 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:49.713436 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:49.713524 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:12:49.714227 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.714771 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.716127 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.716638 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.716744 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:49.716788 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:49.717383 (+ 595us) server_negotiation.cc:299] Negotiation successful
0812 11:12:49.717467 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 11:12:49.717878 9452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81ace18129de47c0a994f1f1d208d0cf" candidate_uuid: "acba0798196d48d691a96b28eaed9b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cb5d354b65e44a2b85e18650db4d504" is_pre_election: true
I20260812 11:12:49.718106 9452 raft_consensus.cc:2454] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acba0798196d48d691a96b28eaed9b11 in term 0.
I20260812 11:12:49.718323 9973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:49.712615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:34241 (local address 127.17.44.195:54309)
0812 11:12:49.713018 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:49.713028 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:49.713252 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:49.713537 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:49.713540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:49.713556 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:12:49.713839 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:49.713870 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.714906 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.714910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:49.715998 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:49.716007 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.716844 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.716848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:49.716942 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:49.717245 (+ 303us) client_negotiation.cc:769] Sending connection context
0812 11:12:49.718110 (+ 865us) client_negotiation.cc:240] Negotiation successful
0812 11:12:49.718193 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":1,"thread_start_us":148,"threads_started":1}
I20260812 11:12:49.718575 9764 leader_election.cc:302] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cb5d354b65e44a2b85e18650db4d504, acba0798196d48d691a96b28eaed9b11; no voters:
I20260812 11:12:49.718902 9970 raft_consensus.cc:2781] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:12:49.718885 9663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81ace18129de47c0a994f1f1d208d0cf" candidate_uuid: "acba0798196d48d691a96b28eaed9b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb17c21d0ed14f238506c985e17099da" is_pre_election: true
I20260812 11:12:49.719018 9970 raft_consensus.cc:482] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:12:49.719063 9970 raft_consensus.cc:3037] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:12:49.719175 9663 raft_consensus.cc:2454] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acba0798196d48d691a96b28eaed9b11 in term 0.
I20260812 11:12:49.720414 9970 raft_consensus.cc:504] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } } peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } }
I20260812 11:12:49.720769 9970 leader_election.cc:288] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11 [CANDIDATE]: Term 1 election: Requested vote from peers 3cb5d354b65e44a2b85e18650db4d504 (127.17.44.193:46467), bb17c21d0ed14f238506c985e17099da (127.17.44.194:34241)
I20260812 11:12:49.721151 9663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81ace18129de47c0a994f1f1d208d0cf" candidate_uuid: "acba0798196d48d691a96b28eaed9b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb17c21d0ed14f238506c985e17099da"
I20260812 11:12:49.721158 9452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81ace18129de47c0a994f1f1d208d0cf" candidate_uuid: "acba0798196d48d691a96b28eaed9b11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cb5d354b65e44a2b85e18650db4d504"
I20260812 11:12:49.721256 9663 raft_consensus.cc:3037] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:12:49.721269 9452 raft_consensus.cc:3037] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:12:49.722260 9958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:49.717797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:44387 (local address 127.17.44.194:34241)
0812 11:12:49.718060 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:49.718064 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:49.718080 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:12:49.718177 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:49.718182 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:49.718230 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:49.718298 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:12:49.718771 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.719448 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.721029 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.721601 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.721684 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:49.721736 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:49.722075 (+ 339us) server_negotiation.cc:299] Negotiation successful
0812 11:12:49.722148 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193}
I20260812 11:12:49.722369 9951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:49.717281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:45793 (local address 127.17.44.195:43595)
0812 11:12:49.717389 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:49.717394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:49.717405 (+ 11us) server_negotiation.cc:407] Connection header received
0812 11:12:49.717441 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:49.717443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:49.717481 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:49.717548 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:12:49.718990 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.719615 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.721133 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.721687 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.721771 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:49.721814 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:49.722169 (+ 355us) server_negotiation.cc:299] Negotiation successful
0812 11:12:49.722237 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 11:12:49.722590 9663 raft_consensus.cc:2454] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acba0798196d48d691a96b28eaed9b11 in term 1.
I20260812 11:12:49.722644 9452 raft_consensus.cc:2454] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acba0798196d48d691a96b28eaed9b11 in term 1.
I20260812 11:12:49.722675 9975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:49.717701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:34241 (local address 127.17.44.193:44387)
0812 11:12:49.717975 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:49.717990 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:49.718085 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:49.718310 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:49.718313 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:49.718437 (+ 124us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:12:49.718634 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:49.718642 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.719590 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.719595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:49.720867 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:49.720878 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.721834 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.721838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:49.721935 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:49.721962 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 11:12:49.722373 (+ 411us) client_negotiation.cc:240] Negotiation successful
0812 11:12:49.722492 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":72,"spinlock_wait_cycles":896,"thread_start_us":96,"threads_started":1}
I20260812 11:12:49.722922 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:49.716937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:43595 (local address 127.17.44.193:45793)
0812 11:12:49.717211 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:49.717225 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:12:49.717312 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:12:49.718390 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:12:49.718392 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:12:49.718415 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:12:49.718808 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:49.718818 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.719864 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.719867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:49.720999 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:12:49.721008 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.721920 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.721923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:12:49.722029 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:49.722053 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 11:12:49.722559 (+ 506us) client_negotiation.cc:240] Negotiation successful
0812 11:12:49.722648 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":372,"spinlock_wait_cycles":15104,"thread_start_us":128,"threads_started":1}
I20260812 11:12:49.723002 9767 leader_election.cc:302] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: acba0798196d48d691a96b28eaed9b11, bb17c21d0ed14f238506c985e17099da; no voters:
I20260812 11:12:49.723160 9970 raft_consensus.cc:2781] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:12:49.723206 9663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81ace18129de47c0a994f1f1d208d0cf" candidate_uuid: "3cb5d354b65e44a2b85e18650db4d504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb17c21d0ed14f238506c985e17099da" is_pre_election: true
I20260812 11:12:49.723380 9663 raft_consensus.cc:2379] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3cb5d354b65e44a2b85e18650db4d504 in current term 1: Already voted for candidate acba0798196d48d691a96b28eaed9b11 in this term.
I20260812 11:12:49.723399 9970 raft_consensus.cc:686] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11 [term 1 LEADER]: Becoming Leader. State: Replica: acba0798196d48d691a96b28eaed9b11, State: Running, Role: LEADER
I20260812 11:12:49.723734 9970 consensus_queue.cc:237] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } } peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } }
I20260812 11:12:49.724104 9862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81ace18129de47c0a994f1f1d208d0cf" candidate_uuid: "3cb5d354b65e44a2b85e18650db4d504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acba0798196d48d691a96b28eaed9b11" is_pre_election: true
I20260812 11:12:49.724725 9349 leader_election.cc:302] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3cb5d354b65e44a2b85e18650db4d504; no voters: acba0798196d48d691a96b28eaed9b11, bb17c21d0ed14f238506c985e17099da
I20260812 11:12:49.724980 9972 raft_consensus.cc:2726] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 11:12:49.728032 9224 catalog_manager.cc:5495] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11 reported cstate change: term changed from 0 to 1, leader changed from <none> to acba0798196d48d691a96b28eaed9b11 (127.17.44.195). New cstate: current_term: 1 leader_uuid: "acba0798196d48d691a96b28eaed9b11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } health_report { overall_health: UNKNOWN } } }
I20260812 11:12:49.803856 9951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:49.799718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47558 (local address 127.17.44.195:43595)
0812 11:12:49.799861 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 11:12:49.799865 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:12:49.799879 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:12:49.799959 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:12:49.799962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:12:49.800020 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:12:49.800116 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:12:49.800452 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.800971 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.801910 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:12:49.802118 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:12:49.802201 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:12:49.802556 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:12:49.803022 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:12:49.803121 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 11:12:49.803472 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 11:12:49.803610 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 11:12:49.811416 9663 raft_consensus.cc:1261] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da [term 1 FOLLOWER]: Refusing update from remote peer acba0798196d48d691a96b28eaed9b11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:12:49.811823 9452 raft_consensus.cc:1261] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Refusing update from remote peer acba0798196d48d691a96b28eaed9b11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:12:49.812419 9977 consensus_queue.cc:1035] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:12:49.812788 9970 consensus_queue.cc:1035] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:12:49.822118 9982 mvcc.cc:205] Tried to move back new op lower bound from 7317639863538536448 to 7317639863191121920. Current Snapshot: MvccSnapshot[applied={T|T < 7317639863538536448}]
I20260812 11:12:49.822656 9983 mvcc.cc:205] Tried to move back new op lower bound from 7317639863538536448 to 7317639863191121920. Current Snapshot: MvccSnapshot[applied={T|T < 7317639863538536448}]
I20260812 11:12:49.832450 9225 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:12:49.832773 9225 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:12:49.835748 9225 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e: Sending DeleteTablet for 3 replicas of tablet 81ace18129de47c0a994f1f1d208d0cf
I20260812 11:12:49.836628 9633 tablet_service.cc:1514] Processing DeleteTablet for tablet 81ace18129de47c0a994f1f1d208d0cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:12:49 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:38430
I20260812 11:12:49.836674 9828 tablet_service.cc:1514] Processing DeleteTablet for tablet 81ace18129de47c0a994f1f1d208d0cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:12:49 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:47548
I20260812 11:12:49.836690 9418 tablet_service.cc:1514] Processing DeleteTablet for tablet 81ace18129de47c0a994f1f1d208d0cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:12:49 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:48838
I20260812 11:12:49.837205 9996 tablet_replica.cc:330] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11: stopping tablet replica
I20260812 11:12:49.837323 9998 tablet_replica.cc:330] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504: stopping tablet replica
I20260812 11:12:49.837359 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 9322
I20260812 11:12:49.844417 9994 tablet_replica.cc:330] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da: stopping tablet replica
I20260812 11:12:49.844803 9994 raft_consensus.cc:2229] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:12:49.845180 9994 raft_consensus.cc:2258] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:12:49.847524 9996 raft_consensus.cc:2229] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:12:49.847941 9996 raft_consensus.cc:2258] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:12:49.849323 9996 ts_tablet_manager.cc:1905] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:12:49.852831 9994 ts_tablet_manager.cc:1905] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:12:49.853271 9996 ts_tablet_manager.cc:1918] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:12:49.853407 9996 log.cc:1196] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/81ace18129de47c0a994f1f1d208d0cf
I20260812 11:12:49.853960 9996 ts_tablet_manager.cc:1939] T 81ace18129de47c0a994f1f1d208d0cf P acba0798196d48d691a96b28eaed9b11: Deleting consensus metadata
I20260812 11:12:49.855638 9206 catalog_manager.cc:4841] TS acba0798196d48d691a96b28eaed9b11 (127.17.44.195:43595): tablet 81ace18129de47c0a994f1f1d208d0cf (table test-table [id=7eb3bd3f564a4cc4be90c348125de32c]) successfully deleted
I20260812 11:12:49.856921 9994 ts_tablet_manager.cc:1918] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260812 11:12:49.856986 9206 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.44.193:46467 (error 108)
I20260812 11:12:49.857153 9994 log.cc:1196] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/81ace18129de47c0a994f1f1d208d0cf
I20260812 11:12:49.857312 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 9527
I20260812 11:12:49.871889 10001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:49.871257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:46467 (local address 127.0.0.1:48844)
0812 11:12:49.871541 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:49.871648 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.193:46467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":116,"threads_started":1}
W20260812 11:12:49.872294 9206 catalog_manager.cc:4584] TS 3cb5d354b65e44a2b85e18650db4d504 (127.17.44.193:46467): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 81ace18129de47c0a994f1f1d208d0cf: Network error: Client connection negotiation failed: client connection to 127.17.44.193:46467: connect: Connection refused (error 111)
I20260812 11:12:49.878746 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 9734
I20260812 11:12:49.878846 10001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:12:49.878482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:34241 (local address 127.0.0.1:38442)
0812 11:12:49.878656 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 11:12:49.878729 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.194:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 11:12:49.894132 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 9187
I20260812 11:13:09.909268 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:42965
--webserver_interface=127.17.44.254
--webserver_port=38247
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:42965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:13:10.050540 10002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:10.050833 10002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:10.050880 10002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:10.055140 10002 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 11:13:10.055203 10002 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 11:13:10.055220 10002 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 11:13:10.055234 10002 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:13:10.055253 10002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:10.055270 10002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:10.055287 10002 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:13:10.055302 10002 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:13:10.061775 10002 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:42965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:42965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=38247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:10.063535 10002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:10.065104 10002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 11:13:10.082335 10002 server_base.cc:1034] running on GCE node
W20260812 11:13:10.082307 10013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:10.082279 10012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:10.082279 10015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:10.083393 10002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:10.084739 10002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:10.085974 10002 hybrid_clock.cc:648] HybridClock initialized: now 1786533190085941 us; error 50 us; skew 500 ppm
Aug 12 11:13:10 dist-test-slave-drl0 krb5kdc[9169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533190, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:13:10.089569 10002 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:13:10.090607 10002 webserver.cc:460] Webserver started at http://127.17.44.254:38247/ using document root <none> and password file <none>
I20260812 11:13:10.091336 10002 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:10.091449 10002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:10.097213 10002 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 11:13:10.100697 10023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:10.102535 10002 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260812 11:13:10.102819 10002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "e11a4e54ab5e486bbc1162221cdcc66e"
format_stamp: "Formatted at 2026-08-12 11:12:48 on dist-test-slave-drl0"
server_key: "d9537b1b73f212dab36936e68f774fb8"
server_key_iv: "f2e95ccf4af2aaa2a51d4dddb7bbe705"
server_key_version: "encryptionkey@0"
I20260812 11:13:10.103597 10002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:10.117154 10002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:10.120865 10002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:10.121197 10002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:10.130811 10002 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:42965
I20260812 11:13:10.131023 10098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:42965 every 8 connection(s)
I20260812 11:13:10.132921 10002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:13:10.139958 10101 sys_catalog.cc:263] Verifying existing consensus state
I20260812 11:13:10.141784 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 10002
I20260812 11:13:10.142271 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:46467
--local_ip_for_outbound_sockets=127.17.44.193
--tserver_master_addrs=127.17.44.254:42965
--webserver_port=40445
--webserver_interface=127.17.44.193
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:13:10.142414 10101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e: Bootstrap starting.
I20260812 11:13:10.162487 10101 log.cc:824] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:10.172514 10101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 11:13:10.172964 10101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e: Bootstrap complete.
I20260812 11:13:10.177317 10101 raft_consensus.cc:348] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11a4e54ab5e486bbc1162221cdcc66e" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42965 } }
I20260812 11:13:10.177736 10101 raft_consensus.cc:729] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e11a4e54ab5e486bbc1162221cdcc66e, State: Initialized, Role: FOLLOWER
I20260812 11:13:10.178309 10101 consensus_queue.cc:260] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11a4e54ab5e486bbc1162221cdcc66e" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42965 } }
I20260812 11:13:10.178743 10101 raft_consensus.cc:388] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:13:10.178879 10101 raft_consensus.cc:482] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:13:10.179064 10101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 1 FOLLOWER]: Advancing to term 2
I20260812 11:13:10.181461 10101 raft_consensus.cc:504] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11a4e54ab5e486bbc1162221cdcc66e" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42965 } }
I20260812 11:13:10.181865 10101 leader_election.cc:302] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e11a4e54ab5e486bbc1162221cdcc66e; no voters:
I20260812 11:13:10.182389 10101 leader_election.cc:288] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 11:13:10.182492 10107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 2 FOLLOWER]: Leader election won for term 2
I20260812 11:13:10.182715 10107 raft_consensus.cc:686] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 2 LEADER]: Becoming Leader. State: Replica: e11a4e54ab5e486bbc1162221cdcc66e, State: Running, Role: LEADER
I20260812 11:13:10.183089 10107 consensus_queue.cc:237] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11a4e54ab5e486bbc1162221cdcc66e" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42965 } }
I20260812 11:13:10.183754 10101 sys_catalog.cc:564] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:13:10.184656 10108 sys_catalog.cc:455] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e11a4e54ab5e486bbc1162221cdcc66e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11a4e54ab5e486bbc1162221cdcc66e" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42965 } } }
I20260812 11:13:10.184662 10109 sys_catalog.cc:455] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e11a4e54ab5e486bbc1162221cdcc66e. Latest consensus state: current_term: 2 leader_uuid: "e11a4e54ab5e486bbc1162221cdcc66e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11a4e54ab5e486bbc1162221cdcc66e" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42965 } } }
I20260812 11:13:10.184813 10109 sys_catalog.cc:458] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [sys.catalog]: This master's current role is: LEADER
I20260812 11:13:10.184810 10108 sys_catalog.cc:458] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [sys.catalog]: This master's current role is: LEADER
I20260812 11:13:10.185276 10118 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:13:10.192101 10118 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:13:10.192832 10118 catalog_manager.cc:1259] Loaded cluster ID: 40fef90bb8a04147982b4aeade7fda17
I20260812 11:13:10.192901 10118 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:13:10.195495 10118 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:13:10.196249 10118 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e: Loaded TSK: 0
I20260812 11:13:10.216939 10118 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e: Generated new TSK 1
I20260812 11:13:10.217392 10118 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 11:13:10.292874 10103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:10.293200 10103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:10.293300 10103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:10.297952 10103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:10.298076 10103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:10.298166 10103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:13:10.303673 10103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:46467
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=40445
--tserver_master_addrs=127.17.44.254:42965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:10.304769 10103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:10.305825 10103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:10.317219 10134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:10.317358 10136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:10.317219 10139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:10.317857 10103 server_base.cc:1034] running on GCE node
I20260812 11:13:10.318274 10103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:10.318991 10103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:10.320202 10103 hybrid_clock.cc:648] HybridClock initialized: now 1786533190320153 us; error 71 us; skew 500 ppm
Aug 12 11:13:10 dist-test-slave-drl0 krb5kdc[9169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533190, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:13:10.328830 10103 init.cc:377] Logged in from keytab as kudu/127.17.44.193@KRBTEST.COM (short username kudu)
I20260812 11:13:10.330634 10103 webserver.cc:460] Webserver started at http://127.17.44.193:40445/ using document root <none> and password file <none>
I20260812 11:13:10.331528 10103 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:10.331630 10103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:10.335948 10103 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:13:10.338637 10152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:10.340241 10103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:13:10.340375 10103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "3cb5d354b65e44a2b85e18650db4d504"
format_stamp: "Formatted at 2026-08-12 11:12:48 on dist-test-slave-drl0"
server_key: "62d8732f984fd8728b64073943ec4d84"
server_key_iv: "bbee7c5fb387d3f02c1d2da362f1d466"
server_key_version: "encryptionkey@0"
I20260812 11:13:10.340967 10103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:10.363858 10103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:10.367631 10103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:10.367939 10103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:10.368625 10103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:13:10.369882 10163 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 11:13:10.374191 10103 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 11:13:10.374260 10103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260812 11:13:10.374377 10103 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 11:13:10.376240 10103 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 11:13:10.376286 10103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260812 11:13:10.376690 10163 tablet_bootstrap.cc:492] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504: Bootstrap starting.
I20260812 11:13:10.388278 10103 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:46467
I20260812 11:13:10.388370 10323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:46467 every 8 connection(s)
I20260812 11:13:10.389322 10103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:13:10.392611 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 10103
I20260812 11:13:10.393231 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:34241
--local_ip_for_outbound_sockets=127.17.44.194
--tserver_master_addrs=127.17.44.254:42965
--webserver_port=46293
--webserver_interface=127.17.44.194
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:13:10.397433 10163 log.cc:824] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504: Log is configured to *not* fsync() on all Append() calls
Aug 12 11:13:10 dist-test-slave-drl0 krb5kdc[9169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533190, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:13:10.408882 10330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:10.391727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.193:33035)
0812 11:13:10.392439 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:10.392493 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:10.393354 (+ 861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:10.395368 (+ 2014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:10.395379 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:10.395954 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:13:10.396624 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:10.396643 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.398517 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.398520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:10.399608 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:10.399621 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.400403 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.400407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:10.400596 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:10.401298 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:13:10.401316 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:13:10.402973 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:13:10.405185 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:10.405191 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:10.405212 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:10.405495 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:10.405876 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:10.405879 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:10.405881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:10.406027 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:10.406640 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:13:10.406646 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:13:10.406898 (+ 252us) client_negotiation.cc:769] Sending connection context
0812 11:13:10.407177 (+ 279us) client_negotiation.cc:240] Negotiation successful
0812 11:13:10.407751 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":158,"threads_started":1}
I20260812 11:13:10.408881 10328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:10.391831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:33035 (local address 127.17.44.254:42965)
0812 11:13:10.392263 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:10.392283 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:10.393666 (+ 1383us) server_negotiation.cc:407] Connection header received
0812 11:13:10.394241 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:10.394258 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:10.394709 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:10.395279 (+ 570us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:10.396881 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.398338 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.399770 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.400240 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.400442 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:10.403195 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:10.403202 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:10.403217 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:13:10.403246 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:10.405046 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:10.405631 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:10.405637 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:10.405641 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:10.405769 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:10.406128 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:10.406133 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:10.406135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:10.406501 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:10.406653 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:10.407495 (+ 842us) server_negotiation.cc:299] Negotiation successful
0812 11:13:10.407845 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":154,"threads_started":1}
I20260812 11:13:10.410600 10163 tablet_bootstrap.cc:492] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 11:13:10.411177 10163 tablet_bootstrap.cc:492] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504: Bootstrap complete.
I20260812 11:13:10.411612 10325 heartbeater.cc:347] Connected to a master server at 127.17.44.254:42965
I20260812 11:13:10.411846 10325 heartbeater.cc:464] Registering TS with master...
I20260812 11:13:10.412384 10325 heartbeater.cc:511] Master 127.17.44.254:42965 requested a full tablet report, sending...
I20260812 11:13:10.412844 10163 ts_tablet_manager.cc:1397] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504: Time spent bootstrapping tablet: real 0.036s user 0.019s sys 0.015s
I20260812 11:13:10.414875 10047 ts_manager.cc:194] Registered new tserver with Master: 3cb5d354b65e44a2b85e18650db4d504 (127.17.44.193:46467)
I20260812 11:13:10.415926 10163 raft_consensus.cc:348] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } } peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } }
I20260812 11:13:10.416461 10163 raft_consensus.cc:729] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cb5d354b65e44a2b85e18650db4d504, State: Initialized, Role: FOLLOWER
I20260812 11:13:10.416947 10163 consensus_queue.cc:260] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } } peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } }
I20260812 11:13:10.417992 10163 ts_tablet_manager.cc:1428] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260812 11:13:10.419565 10047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.193@KRBTEST.COM'} at 127.17.44.193:33035
I20260812 11:13:10.422219 10325 heartbeater.cc:503] Master 127.17.44.254:42965 was elected leader, sending a full tablet report...
Aug 12 11:13:10 dist-test-slave-drl0 krb5kdc[9169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533190, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for kudu/127.17.44.193@KRBTEST.COM
I20260812 11:13:10.434758 10338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:10.418855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:46467 (local address 127.0.0.1:35468)
0812 11:13:10.419470 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:10.419519 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:10.419677 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:10.420873 (+ 1196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:10.420879 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:10.421326 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:13:10.421467 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:10.421476 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.422748 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.422751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:10.423459 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:10.423469 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.424223 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.424226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:10.424337 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:10.424908 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:13:10.424920 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:13:10.426299 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:13:10.429517 (+ 3218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:10.429528 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:10.429534 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:10.429771 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:10.431077 (+ 1306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:10.431084 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:10.431087 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:10.431273 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:10.432032 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:13:10.432041 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:13:10.432205 (+ 164us) client_negotiation.cc:769] Sending connection context
0812 11:13:10.432383 (+ 178us) client_negotiation.cc:240] Negotiation successful
0812 11:13:10.433519 (+ 1136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":150,"threads_started":1}
I20260812 11:13:10.434758 10340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:10.419243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35468 (local address 127.17.44.193:46467)
0812 11:13:10.420411 (+ 1168us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:10.420421 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:10.420440 (+ 19us) server_negotiation.cc:407] Connection header received
0812 11:13:10.420493 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:10.420497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:10.420709 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:10.420847 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:10.421608 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.422591 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.423592 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.424092 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.424228 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:10.426439 (+ 2211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:10.426449 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:10.426462 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:13:10.426495 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:10.429387 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:10.429915 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:10.429920 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:10.429922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:10.430013 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:10.431442 (+ 1429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:10.431447 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:10.431449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:10.431849 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:10.432047 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:10.432522 (+ 475us) server_negotiation.cc:299] Negotiation successful
0812 11:13:10.433530 (+ 1008us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":1065,"thread_start_us":131,"threads_started":1}
I20260812 11:13:10.437188 10226 tablet_service.cc:1514] Processing DeleteTablet for tablet 81ace18129de47c0a994f1f1d208d0cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:12:49 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:35468
I20260812 11:13:10.438455 10341 tablet_replica.cc:330] stopping tablet replica
I20260812 11:13:10.438987 10341 raft_consensus.cc:2229] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:13:10.439301 10341 raft_consensus.cc:2258] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:13:10.440448 10341 ts_tablet_manager.cc:1905] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:13:10.444515 10341 ts_tablet_manager.cc:1918] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:13:10.444604 10341 log.cc:1196] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/wal/wals/81ace18129de47c0a994f1f1d208d0cf
I20260812 11:13:10.444911 10341 ts_tablet_manager.cc:1939] T 81ace18129de47c0a994f1f1d208d0cf P 3cb5d354b65e44a2b85e18650db4d504: Deleting consensus metadata
I20260812 11:13:10.447197 10026 catalog_manager.cc:4841] TS 3cb5d354b65e44a2b85e18650db4d504 (127.17.44.193:46467): tablet 81ace18129de47c0a994f1f1d208d0cf (table test-table [id=7eb3bd3f564a4cc4be90c348125de32c]) successfully deleted
W20260812 11:13:10.532228 10331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:10.532508 10331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:10.532584 10331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:10.536562 10331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:10.536649 10331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:10.536738 10331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:13:10.543210 10331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:34241
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=46293
--tserver_master_addrs=127.17.44.254:42965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:10.545116 10331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:10.546801 10331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:10.557526 10352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:10.557549 10348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:10.557545 10347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:10.557930 10331 server_base.cc:1034] running on GCE node
I20260812 11:13:10.558440 10331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:10.559106 10331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:10.560477 10331 hybrid_clock.cc:648] HybridClock initialized: now 1786533190560455 us; error 58 us; skew 500 ppm
Aug 12 11:13:10 dist-test-slave-drl0 krb5kdc[9169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533190, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:13:10.563983 10331 init.cc:377] Logged in from keytab as kudu/127.17.44.194@KRBTEST.COM (short username kudu)
I20260812 11:13:10.564965 10331 webserver.cc:460] Webserver started at http://127.17.44.194:46293/ using document root <none> and password file <none>
I20260812 11:13:10.565508 10331 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:10.565593 10331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:10.569867 10331 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 11:13:10.572356 10365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:10.573706 10331 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 11:13:10.573858 10331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "bb17c21d0ed14f238506c985e17099da"
format_stamp: "Formatted at 2026-08-12 11:12:48 on dist-test-slave-drl0"
server_key: "9827b9bd2d0cfec07cff7cf005a35042"
server_key_iv: "c4e7559908d10abef0b580a987ccd072"
server_key_version: "encryptionkey@0"
I20260812 11:13:10.574415 10331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:10.602188 10331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:10.605885 10331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:10.606099 10331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:10.606711 10331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:13:10.608278 10376 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260812 11:13:10.613307 10376 ts_tablet_manager.cc:1852] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260812 11:13:10.613698 10376 ts_tablet_manager.cc:1905] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:13:10.618036 10376 ts_tablet_manager.cc:1918] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:13:10.618134 10376 log.cc:1196] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/81ace18129de47c0a994f1f1d208d0cf
I20260812 11:13:10.618552 10376 ts_tablet_manager.cc:1939] T 81ace18129de47c0a994f1f1d208d0cf P bb17c21d0ed14f238506c985e17099da: Deleting consensus metadata
I20260812 11:13:10.618909 10331 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260812 11:13:10.619074 10331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.011s user 0.000s sys 0.000s
I20260812 11:13:10.619158 10331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:13:10.619201 10331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:10.628975 10331 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:34241
I20260812 11:13:10.629032 10537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:34241 every 8 connection(s)
I20260812 11:13:10.629977 10331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:13:10.635007 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 10331
I20260812 11:13:10.635578 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:43595
--local_ip_for_outbound_sockets=127.17.44.195
--tserver_master_addrs=127.17.44.254:42965
--webserver_port=34383
--webserver_interface=127.17.44.195
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 11:13:10 dist-test-slave-drl0 krb5kdc[9169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533190, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:13:10.646644 10328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:10.632173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:41319 (local address 127.17.44.254:42965)
0812 11:13:10.632360 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:10.632364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:10.633467 (+ 1103us) server_negotiation.cc:407] Connection header received
0812 11:13:10.634905 (+ 1438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:10.634909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:10.635152 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:10.635318 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:10.636606 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.637145 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.639066 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.639380 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.639529 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:10.642810 (+ 3281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:10.642813 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:10.642815 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:13:10.642843 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:10.644364 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:10.644947 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:10.644952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:10.644954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:10.645036 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:10.645487 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:10.645492 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:10.645494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:10.645689 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:10.645794 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:10.646312 (+ 518us) server_negotiation.cc:299] Negotiation successful
0812 11:13:10.646466 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 11:13:10.647603 10545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:10.632404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.194:41319)
0812 11:13:10.633298 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:10.633346 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:10.634509 (+ 1163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:10.635500 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:10.635508 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:10.635965 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:13:10.636442 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:10.636457 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.637278 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.637282 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:10.638707 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:10.638722 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.639605 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.639612 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:10.639856 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:10.640666 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:13:10.640684 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:13:10.642618 (+ 1934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:13:10.644526 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:10.644532 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:10.644547 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:10.644829 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:10.645139 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:10.645142 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:10.645144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:10.645358 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:10.645807 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:13:10.645815 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:13:10.646073 (+ 258us) client_negotiation.cc:769] Sending connection context
0812 11:13:10.646269 (+ 196us) client_negotiation.cc:240] Negotiation successful
0812 11:13:10.646593 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":124,"threads_started":1}
I20260812 11:13:10.648880 10540 heartbeater.cc:347] Connected to a master server at 127.17.44.254:42965
I20260812 11:13:10.649148 10540 heartbeater.cc:464] Registering TS with master...
I20260812 11:13:10.649652 10540 heartbeater.cc:511] Master 127.17.44.254:42965 requested a full tablet report, sending...
I20260812 11:13:10.651083 10047 ts_manager.cc:194] Registered new tserver with Master: bb17c21d0ed14f238506c985e17099da (127.17.44.194:34241)
I20260812 11:13:10.652256 10047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.194@KRBTEST.COM'} at 127.17.44.194:41319
W20260812 11:13:10.767268 10546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:10.768162 10546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:10.768306 10546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:10.773275 10546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:10.773352 10546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:10.773425 10546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:13:10.779139 10546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:43595
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=34383
--tserver_master_addrs=127.17.44.254:42965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:10.780254 10546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:10.781374 10546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:10.792342 10554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:10.792411 10557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:10.792366 10559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:10.792672 10546 server_base.cc:1034] running on GCE node
I20260812 11:13:10.793615 10546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:10.794373 10546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:10.795563 10546 hybrid_clock.cc:648] HybridClock initialized: now 1786533190795533 us; error 44 us; skew 500 ppm
Aug 12 11:13:10 dist-test-slave-drl0 krb5kdc[9169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533190, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:13:10.798714 10546 init.cc:377] Logged in from keytab as kudu/127.17.44.195@KRBTEST.COM (short username kudu)
I20260812 11:13:10.799680 10546 webserver.cc:460] Webserver started at http://127.17.44.195:34383/ using document root <none> and password file <none>
I20260812 11:13:10.800223 10546 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:10.800305 10546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:10.804476 10546 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:13:10.807160 10572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:10.808735 10546 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:13:10.808842 10546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "acba0798196d48d691a96b28eaed9b11"
format_stamp: "Formatted at 2026-08-12 11:12:49 on dist-test-slave-drl0"
server_key: "f7883bfd4ed9853bab4033de0aaeca3b"
server_key_iv: "498fb7c6b4df3389e041801c76ac2a35"
server_key_version: "encryptionkey@0"
I20260812 11:13:10.809427 10546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:10.822752 10546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:10.826274 10546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:10.826516 10546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:10.827211 10546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:13:10.828058 10546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:13:10.828106 10546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:10.828145 10546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:13:10.828161 10546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:10.840354 10546 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:43595
I20260812 11:13:10.840376 10721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:43595 every 8 connection(s)
I20260812 11:13:10.841452 10546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:13:10.845413 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 10546
Aug 12 11:13:10 dist-test-slave-drl0 krb5kdc[9169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533190, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:13:10.856066 10725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:10.846212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35318 (local address 127.17.44.254:42965)
0812 11:13:10.846673 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:10.846676 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:10.846690 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:13:10.846746 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:10.846748 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:10.846784 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:10.846855 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:10.847848 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.848408 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.849345 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.849674 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.849819 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:10.851400 (+ 1581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:10.851403 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:10.851405 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:13:10.851427 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:10.853504 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:10.854131 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:10.854135 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:10.854136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:10.854253 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:10.854587 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:10.854593 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:10.854597 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:10.854863 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:10.855145 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:10.855664 (+ 519us) server_negotiation.cc:299] Negotiation successful
0812 11:13:10.855923 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":101,"threads_started":1}
I20260812 11:13:10.859807 10328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:10.843552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:34183 (local address 127.17.44.254:42965)
0812 11:13:10.843731 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:10.843735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:10.845336 (+ 1601us) server_negotiation.cc:407] Connection header received
0812 11:13:10.846560 (+ 1224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:10.846565 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:10.846630 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:10.846687 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:10.848422 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.848914 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.850470 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.850683 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.850828 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:10.854333 (+ 3505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:10.854337 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:10.854340 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:13:10.854368 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:10.856935 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:10.857708 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:10.857712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:10.857713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:10.857822 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:10.858496 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:10.858502 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:10.858508 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:10.858791 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:10.858913 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:10.859523 (+ 610us) server_negotiation.cc:299] Negotiation successful
0812 11:13:10.859648 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 11:13:10.860699 10724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:10.843899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.195:34183)
0812 11:13:10.845108 (+ 1209us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:10.845190 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:10.846294 (+ 1104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:10.847184 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:10.847197 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:10.847698 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:13:10.848278 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:10.848292 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.849047 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.849051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:10.850063 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:10.850073 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.850874 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.850879 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:10.851102 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:10.851714 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:13:10.851738 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:13:10.854105 (+ 2367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:13:10.857073 (+ 2968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:10.857080 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:10.857101 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:10.857517 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:10.857994 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:10.858001 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:10.858008 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:10.858309 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:10.858914 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:13:10.858920 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:13:10.859256 (+ 336us) client_negotiation.cc:769] Sending connection context
0812 11:13:10.859479 (+ 223us) client_negotiation.cc:240] Negotiation successful
0812 11:13:10.859840 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":992,"thread_start_us":631,"threads_started":1}
I20260812 11:13:10.862002 10722 heartbeater.cc:347] Connected to a master server at 127.17.44.254:42965
I20260812 11:13:10.862231 10722 heartbeater.cc:464] Registering TS with master...
I20260812 11:13:10.862696 10722 heartbeater.cc:511] Master 127.17.44.254:42965 requested a full tablet report, sending...
I20260812 11:13:10.864006 10047 ts_manager.cc:194] Registered new tserver with Master: acba0798196d48d691a96b28eaed9b11 (127.17.44.195:43595)
I20260812 11:13:10.864775 10047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.195@KRBTEST.COM'} at 127.17.44.195:34183
I20260812 11:13:10.871387 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:13:10.876402 10328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:10.872360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35322 (local address 127.17.44.254:42965)
0812 11:13:10.872553 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:10.872558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:10.872579 (+ 21us) server_negotiation.cc:407] Connection header received
0812 11:13:10.872792 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:10.872795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:10.872943 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:10.873083 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:13:10.873381 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.873987 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.875118 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.875363 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.875478 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:10.875814 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:13:10.875968 (+ 154us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 11:13:10.876263 (+ 295us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35322: token expired
Metrics: {"server-negotiator.queue_time_us":62}
W20260812 11:13:10.876492 10328 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35322: token expired
W20260812 11:13:10.877116 9937 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.17.44.254:42965)
I20260812 11:13:10.887081 10328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:10.877690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35330 (local address 127.17.44.254:42965)
0812 11:13:10.877843 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:10.877849 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:10.877870 (+ 21us) server_negotiation.cc:407] Connection header received
0812 11:13:10.878075 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:10.878081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:10.878146 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:10.878242 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:10.878998 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.879510 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.880445 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.880650 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.880739 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:10.882702 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:10.882709 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:10.882713 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:13:10.882744 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:10.884336 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:10.884839 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:10.884844 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:10.884845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:10.885066 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:10.885591 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:10.885594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:10.885596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:10.885949 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:10.886065 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:10.886493 (+ 428us) server_negotiation.cc:299] Negotiation successful
0812 11:13:10.886592 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 11:13:10.891149 10047 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:13:10.894749 10047 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:13:10.904906 10226 tablet_service.cc:1467] Processing CreateTablet for tablet f32e1c829fc0424ab9c3e539391ad080 (DEFAULT_TABLE table=test-table [id=e9421a3f69cc48b788cb8a0fa0909aa6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:13:10.905602 10226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f32e1c829fc0424ab9c3e539391ad080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:10.909186 10734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:10.903558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:43595 (local address 127.0.0.1:49342)
0812 11:13:10.903843 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:10.903858 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:10.903976 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:10.904760 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:10.904763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:10.904781 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:10.904864 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:10.904870 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.906419 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.906422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:10.907684 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:10.907694 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.908702 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.908705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:10.908791 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:10.908834 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 11:13:10.908943 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 11:13:10.909038 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":122,"threads_started":1}
I20260812 11:13:10.909600 10163 tablet_bootstrap.cc:492] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504: Bootstrap starting.
I20260812 11:13:10.909633 10738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:10.903776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49342 (local address 127.17.44.195:43595)
0812 11:13:10.904266 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:10.904277 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:10.904298 (+ 21us) server_negotiation.cc:407] Connection header received
0812 11:13:10.904379 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:10.904384 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:10.904589 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:10.904742 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:10.905133 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.906297 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.907838 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.908569 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.908689 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:10.908839 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:10.909373 (+ 534us) server_negotiation.cc:299] Negotiation successful
0812 11:13:10.909477 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":165,"threads_started":1}
I20260812 11:13:10.910564 10338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:10.903305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:34241 (local address 127.0.0.1:60020)
0812 11:13:10.903444 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:10.903460 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:10.903569 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:10.904560 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:10.904562 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:10.904580 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:10.904712 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:10.904718 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.906168 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.906174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:10.907641 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:10.907653 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.910148 (+ 2495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.910151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:10.910248 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:10.910269 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 11:13:10.910364 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 11:13:10.910443 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":34}
I20260812 11:13:10.910634 10736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:10.903540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60020 (local address 127.17.44.194:34241)
0812 11:13:10.904024 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:10.904038 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:10.904065 (+ 27us) server_negotiation.cc:407] Connection header received
0812 11:13:10.904152 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:10.904160 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:10.904373 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:10.904568 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:10.904863 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.906051 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.909197 (+ 3146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:10.910012 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:10.910135 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:10.910264 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:10.910397 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 11:13:10.910478 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":154,"threads_started":1}
I20260812 11:13:10.910681 10163 tablet_bootstrap.cc:654] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:10.912046 10163 tablet_bootstrap.cc:492] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504: No bootstrap required, opened a new log
I20260812 11:13:10.912299 10163 ts_tablet_manager.cc:1397] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 11:13:10.912722 10163 raft_consensus.cc:348] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } } peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } }
I20260812 11:13:10.912864 10163 raft_consensus.cc:374] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:10.912905 10163 raft_consensus.cc:729] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cb5d354b65e44a2b85e18650db4d504, State: Initialized, Role: FOLLOWER
I20260812 11:13:10.913043 10163 consensus_queue.cc:260] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } } peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } }
I20260812 11:13:10.913409 10163 ts_tablet_manager.cc:1428] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 11:13:10.913806 10635 tablet_service.cc:1467] Processing CreateTablet for tablet f32e1c829fc0424ab9c3e539391ad080 (DEFAULT_TABLE table=test-table [id=e9421a3f69cc48b788cb8a0fa0909aa6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:13:10.914821 10635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f32e1c829fc0424ab9c3e539391ad080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:10.917693 10441 tablet_service.cc:1467] Processing CreateTablet for tablet f32e1c829fc0424ab9c3e539391ad080 (DEFAULT_TABLE table=test-table [id=e9421a3f69cc48b788cb8a0fa0909aa6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:13:10.919149 10441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f32e1c829fc0424ab9c3e539391ad080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:10.923180 10740 tablet_bootstrap.cc:492] T f32e1c829fc0424ab9c3e539391ad080 P acba0798196d48d691a96b28eaed9b11: Bootstrap starting.
I20260812 11:13:10.923683 10376 tablet_bootstrap.cc:492] T f32e1c829fc0424ab9c3e539391ad080 P bb17c21d0ed14f238506c985e17099da: Bootstrap starting.
I20260812 11:13:10.925668 10376 tablet_bootstrap.cc:654] T f32e1c829fc0424ab9c3e539391ad080 P bb17c21d0ed14f238506c985e17099da: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:10.926088 10740 tablet_bootstrap.cc:654] T f32e1c829fc0424ab9c3e539391ad080 P acba0798196d48d691a96b28eaed9b11: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:10.926483 10376 log.cc:824] T f32e1c829fc0424ab9c3e539391ad080 P bb17c21d0ed14f238506c985e17099da: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:10.926793 10740 log.cc:824] T f32e1c829fc0424ab9c3e539391ad080 P acba0798196d48d691a96b28eaed9b11: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:10.928709 10376 tablet_bootstrap.cc:492] T f32e1c829fc0424ab9c3e539391ad080 P bb17c21d0ed14f238506c985e17099da: No bootstrap required, opened a new log
I20260812 11:13:10.928921 10376 ts_tablet_manager.cc:1397] T f32e1c829fc0424ab9c3e539391ad080 P bb17c21d0ed14f238506c985e17099da: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 11:13:10.929049 10740 tablet_bootstrap.cc:492] T f32e1c829fc0424ab9c3e539391ad080 P acba0798196d48d691a96b28eaed9b11: No bootstrap required, opened a new log
I20260812 11:13:10.929258 10740 ts_tablet_manager.cc:1397] T f32e1c829fc0424ab9c3e539391ad080 P acba0798196d48d691a96b28eaed9b11: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 11:13:10.933177 10376 raft_consensus.cc:348] T f32e1c829fc0424ab9c3e539391ad080 P bb17c21d0ed14f238506c985e17099da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } } peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } }
I20260812 11:13:10.933456 10376 raft_consensus.cc:374] T f32e1c829fc0424ab9c3e539391ad080 P bb17c21d0ed14f238506c985e17099da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:10.933501 10376 raft_consensus.cc:729] T f32e1c829fc0424ab9c3e539391ad080 P bb17c21d0ed14f238506c985e17099da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb17c21d0ed14f238506c985e17099da, State: Initialized, Role: FOLLOWER
I20260812 11:13:10.933944 10376 consensus_queue.cc:260] T f32e1c829fc0424ab9c3e539391ad080 P bb17c21d0ed14f238506c985e17099da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } } peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } }
I20260812 11:13:10.934983 10376 ts_tablet_manager.cc:1428] T f32e1c829fc0424ab9c3e539391ad080 P bb17c21d0ed14f238506c985e17099da: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260812 11:13:10.934964 10740 raft_consensus.cc:348] T f32e1c829fc0424ab9c3e539391ad080 P acba0798196d48d691a96b28eaed9b11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } } peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } }
I20260812 11:13:10.935585 10740 raft_consensus.cc:374] T f32e1c829fc0424ab9c3e539391ad080 P acba0798196d48d691a96b28eaed9b11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:10.935715 10740 raft_consensus.cc:729] T f32e1c829fc0424ab9c3e539391ad080 P acba0798196d48d691a96b28eaed9b11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acba0798196d48d691a96b28eaed9b11, State: Initialized, Role: FOLLOWER
I20260812 11:13:10.936384 10740 consensus_queue.cc:260] T f32e1c829fc0424ab9c3e539391ad080 P acba0798196d48d691a96b28eaed9b11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } } peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } }
I20260812 11:13:10.937278 10722 heartbeater.cc:503] Master 127.17.44.254:42965 was elected leader, sending a full tablet report...
I20260812 11:13:10.937551 10740 ts_tablet_manager.cc:1428] T f32e1c829fc0424ab9c3e539391ad080 P acba0798196d48d691a96b28eaed9b11: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
I20260812 11:13:10.942179 10540 heartbeater.cc:503] Master 127.17.44.254:42965 was elected leader, sending a full tablet report...
W20260812 11:13:11.092555 10723 tablet.cc:2369] T f32e1c829fc0424ab9c3e539391ad080 P acba0798196d48d691a96b28eaed9b11: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:13:11.128484 10334 raft_consensus.cc:482] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:13:11.128890 10334 raft_consensus.cc:504] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } } peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } }
I20260812 11:13:11.130438 10334 leader_election.cc:288] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb17c21d0ed14f238506c985e17099da (127.17.44.194:34241), acba0798196d48d691a96b28eaed9b11 (127.17.44.195:43595)
W20260812 11:13:11.131278 10544 tablet.cc:2369] T f32e1c829fc0424ab9c3e539391ad080 P bb17c21d0ed14f238506c985e17099da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:13:11.136386 10736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:11.130816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:37793 (local address 127.17.44.194:34241)
0812 11:13:11.130977 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:11.130982 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:11.131114 (+ 132us) server_negotiation.cc:407] Connection header received
0812 11:13:11.131474 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:11.131477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:11.131541 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:11.131591 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:11.132064 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:11.132731 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:11.134513 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:11.135346 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:11.135535 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:11.135617 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:11.136115 (+ 498us) server_negotiation.cc:299] Negotiation successful
0812 11:13:11.136239 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 11:13:11.136514 10750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:11.130614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:34241 (local address 127.17.44.193:37793)
0812 11:13:11.131020 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:11.131036 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:11.131214 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:11.131758 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:11.131761 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:11.131792 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:11.131929 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:11.131936 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:11.132900 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:11.132904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:11.134346 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:11.134366 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:11.135553 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:11.135560 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:11.135687 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:11.135889 (+ 202us) client_negotiation.cc:769] Sending connection context
0812 11:13:11.136167 (+ 278us) client_negotiation.cc:240] Negotiation successful
0812 11:13:11.136308 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":77,"thread_start_us":119,"threads_started":1}
I20260812 11:13:11.136880 10738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:11.131571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:44465 (local address 127.17.44.195:43595)
0812 11:13:11.131727 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:11.131733 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:11.131747 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:13:11.131796 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:11.131798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:11.131842 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:11.131913 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:11.132331 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:11.133084 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:11.135207 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:11.136070 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:11.136217 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:11.136286 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:11.136647 (+ 361us) server_negotiation.cc:299] Negotiation successful
0812 11:13:11.136758 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 11:13:11.137302 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:11.130731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:43595 (local address 127.17.44.193:44465)
0812 11:13:11.131138 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:11.131151 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:11.131217 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:11.132072 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:11.132074 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:11.132093 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:11.132169 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:11.132175 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:11.133249 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:11.133252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:11.134875 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:11.134900 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:11.136359 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:11.136363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:11.136474 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:11.136503 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 11:13:11.137026 (+ 523us) client_negotiation.cc:240] Negotiation successful
0812 11:13:11.137149 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":61,"thread_start_us":73,"threads_started":1}
I20260812 11:13:11.137368 10472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f32e1c829fc0424ab9c3e539391ad080" candidate_uuid: "3cb5d354b65e44a2b85e18650db4d504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb17c21d0ed14f238506c985e17099da" is_pre_election: true
I20260812 11:13:11.137703 10472 raft_consensus.cc:2454] T f32e1c829fc0424ab9c3e539391ad080 P bb17c21d0ed14f238506c985e17099da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3cb5d354b65e44a2b85e18650db4d504 in term 0.
I20260812 11:13:11.137941 10658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f32e1c829fc0424ab9c3e539391ad080" candidate_uuid: "3cb5d354b65e44a2b85e18650db4d504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acba0798196d48d691a96b28eaed9b11" is_pre_election: true
I20260812 11:13:11.138193 10658 raft_consensus.cc:2454] T f32e1c829fc0424ab9c3e539391ad080 P acba0798196d48d691a96b28eaed9b11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3cb5d354b65e44a2b85e18650db4d504 in term 0.
I20260812 11:13:11.138311 10156 leader_election.cc:302] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cb5d354b65e44a2b85e18650db4d504, bb17c21d0ed14f238506c985e17099da; no voters:
I20260812 11:13:11.138667 10334 raft_consensus.cc:2781] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:13:11.138859 10334 raft_consensus.cc:482] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:13:11.138970 10334 raft_consensus.cc:3037] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:11.140445 10334 raft_consensus.cc:504] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } } peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } }
I20260812 11:13:11.140878 10334 leader_election.cc:288] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 1 election: Requested vote from peers bb17c21d0ed14f238506c985e17099da (127.17.44.194:34241), acba0798196d48d691a96b28eaed9b11 (127.17.44.195:43595)
W20260812 11:13:11.140967 10327 tablet.cc:2369] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:13:11.141278 10472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f32e1c829fc0424ab9c3e539391ad080" candidate_uuid: "3cb5d354b65e44a2b85e18650db4d504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb17c21d0ed14f238506c985e17099da"
I20260812 11:13:11.141332 10658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f32e1c829fc0424ab9c3e539391ad080" candidate_uuid: "3cb5d354b65e44a2b85e18650db4d504" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acba0798196d48d691a96b28eaed9b11"
I20260812 11:13:11.141420 10472 raft_consensus.cc:3037] T f32e1c829fc0424ab9c3e539391ad080 P bb17c21d0ed14f238506c985e17099da [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:11.141443 10658 raft_consensus.cc:3037] T f32e1c829fc0424ab9c3e539391ad080 P acba0798196d48d691a96b28eaed9b11 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:11.142722 10472 raft_consensus.cc:2454] T f32e1c829fc0424ab9c3e539391ad080 P bb17c21d0ed14f238506c985e17099da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3cb5d354b65e44a2b85e18650db4d504 in term 1.
I20260812 11:13:11.142722 10658 raft_consensus.cc:2454] T f32e1c829fc0424ab9c3e539391ad080 P acba0798196d48d691a96b28eaed9b11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3cb5d354b65e44a2b85e18650db4d504 in term 1.
I20260812 11:13:11.143128 10154 leader_election.cc:302] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cb5d354b65e44a2b85e18650db4d504, acba0798196d48d691a96b28eaed9b11; no voters:
I20260812 11:13:11.143405 10334 raft_consensus.cc:2781] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:13:11.143639 10334 raft_consensus.cc:686] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 1 LEADER]: Becoming Leader. State: Replica: 3cb5d354b65e44a2b85e18650db4d504, State: Running, Role: LEADER
I20260812 11:13:11.143904 10334 consensus_queue.cc:237] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } } peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } }
I20260812 11:13:11.147245 10044 catalog_manager.cc:5495] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3cb5d354b65e44a2b85e18650db4d504 (127.17.44.193). New cstate: current_term: 1 leader_uuid: "3cb5d354b65e44a2b85e18650db4d504" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } health_report { overall_health: HEALTHY } } }
I20260812 11:13:11.167296 10755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:11.163015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35470 (local address 127.17.44.193:46467)
0812 11:13:11.163362 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:11.163369 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:11.163383 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:13:11.163455 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:11.163457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:11.163509 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:11.163619 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:13:11.163914 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:11.164479 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:11.165331 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:11.165587 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:11.165795 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:11.166304 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:13:11.166729 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:13:11.166838 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:11.167072 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 11:13:11.167126 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":146,"threads_started":1}
I20260812 11:13:11.172668 10472 raft_consensus.cc:1261] T f32e1c829fc0424ab9c3e539391ad080 P bb17c21d0ed14f238506c985e17099da [term 1 FOLLOWER]: Refusing update from remote peer 3cb5d354b65e44a2b85e18650db4d504: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:13:11.172686 10658 raft_consensus.cc:1261] T f32e1c829fc0424ab9c3e539391ad080 P acba0798196d48d691a96b28eaed9b11 [term 1 FOLLOWER]: Refusing update from remote peer 3cb5d354b65e44a2b85e18650db4d504: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:13:11.173694 10334 consensus_queue.cc:1035] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:13:11.174172 10751 consensus_queue.cc:1035] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:13:11.181604 10757 mvcc.cc:205] Tried to move back new op lower bound from 7317639951032524800 to 7317639950928228352. Current Snapshot: MvccSnapshot[applied={T|T < 7317639951032524800}]
I20260812 11:13:11.183974 10759 mvcc.cc:205] Tried to move back new op lower bound from 7317639951032524800 to 7317639950928228352. Current Snapshot: MvccSnapshot[applied={T|T < 7317639951032524800}]
I20260812 11:13:11.190290 10047 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:13:11.190521 10047 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:13:11.193712 10047 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e: Sending DeleteTablet for 3 replicas of tablet f32e1c829fc0424ab9c3e539391ad080
I20260812 11:13:11.194725 10441 tablet_service.cc:1514] Processing DeleteTablet for tablet f32e1c829fc0424ab9c3e539391ad080 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:13:11 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:60020
I20260812 11:13:11.194818 10226 tablet_service.cc:1514] Processing DeleteTablet for tablet f32e1c829fc0424ab9c3e539391ad080 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:13:11 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:35468
I20260812 11:13:11.194829 10635 tablet_service.cc:1514] Processing DeleteTablet for tablet f32e1c829fc0424ab9c3e539391ad080 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:13:11 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:49342
I20260812 11:13:11.195299 10769 tablet_replica.cc:330] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504: stopping tablet replica
I20260812 11:13:11.195628 10771 tablet_replica.cc:330] T f32e1c829fc0424ab9c3e539391ad080 P acba0798196d48d691a96b28eaed9b11: stopping tablet replica
I20260812 11:13:11.195636 10769 raft_consensus.cc:2229] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:13:11.195763 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 10103
I20260812 11:13:11.196008 10771 raft_consensus.cc:2229] T f32e1c829fc0424ab9c3e539391ad080 P acba0798196d48d691a96b28eaed9b11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:13:11.196058 10769 raft_consensus.cc:2258] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:13:11.196341 10771 raft_consensus.cc:2258] T f32e1c829fc0424ab9c3e539391ad080 P acba0798196d48d691a96b28eaed9b11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:13:11.200038 10772 tablet_replica.cc:330] T f32e1c829fc0424ab9c3e539391ad080 P bb17c21d0ed14f238506c985e17099da: stopping tablet replica
I20260812 11:13:11.200538 10772 raft_consensus.cc:2229] T f32e1c829fc0424ab9c3e539391ad080 P bb17c21d0ed14f238506c985e17099da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:13:11.200960 10772 raft_consensus.cc:2258] T f32e1c829fc0424ab9c3e539391ad080 P bb17c21d0ed14f238506c985e17099da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:13:11.203739 10771 ts_tablet_manager.cc:1905] T f32e1c829fc0424ab9c3e539391ad080 P acba0798196d48d691a96b28eaed9b11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:13:11.204092 10772 ts_tablet_manager.cc:1905] T f32e1c829fc0424ab9c3e539391ad080 P bb17c21d0ed14f238506c985e17099da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:13:11.206588 10771 ts_tablet_manager.cc:1918] T f32e1c829fc0424ab9c3e539391ad080 P acba0798196d48d691a96b28eaed9b11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:13:11.206718 10771 log.cc:1196] T f32e1c829fc0424ab9c3e539391ad080 P acba0798196d48d691a96b28eaed9b11: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/f32e1c829fc0424ab9c3e539391ad080
I20260812 11:13:11.207130 10771 ts_tablet_manager.cc:1939] T f32e1c829fc0424ab9c3e539391ad080 P acba0798196d48d691a96b28eaed9b11: Deleting consensus metadata
I20260812 11:13:11.207393 10772 ts_tablet_manager.cc:1918] T f32e1c829fc0424ab9c3e539391ad080 P bb17c21d0ed14f238506c985e17099da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:13:11.207535 10772 log.cc:1196] T f32e1c829fc0424ab9c3e539391ad080 P bb17c21d0ed14f238506c985e17099da: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/f32e1c829fc0424ab9c3e539391ad080
I20260812 11:13:11.207870 10772 ts_tablet_manager.cc:1939] T f32e1c829fc0424ab9c3e539391ad080 P bb17c21d0ed14f238506c985e17099da: Deleting consensus metadata
I20260812 11:13:11.208772 10026 catalog_manager.cc:4841] TS acba0798196d48d691a96b28eaed9b11 (127.17.44.195:43595): tablet f32e1c829fc0424ab9c3e539391ad080 (table test-table [id=e9421a3f69cc48b788cb8a0fa0909aa6]) successfully deleted
I20260812 11:13:11.209390 10028 catalog_manager.cc:4841] TS bb17c21d0ed14f238506c985e17099da (127.17.44.194:34241): tablet f32e1c829fc0424ab9c3e539391ad080 (table test-table [id=e9421a3f69cc48b788cb8a0fa0909aa6]) successfully deleted
W20260812 11:13:11.212903 10026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.44.193:46467 (error 108)
I20260812 11:13:11.213903 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 10331
I20260812 11:13:11.214082 10338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:11.213744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:46467 (local address 127.0.0.1:35482)
0812 11:13:11.213879 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:11.213969 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.193:46467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 11:13:11.217705 10026 catalog_manager.cc:4584] TS 3cb5d354b65e44a2b85e18650db4d504 (127.17.44.193:46467): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f32e1c829fc0424ab9c3e539391ad080: Network error: Client connection negotiation failed: client connection to 127.17.44.193:46467: connect: Connection refused (error 111)
I20260812 11:13:11.243388 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 10546
I20260812 11:13:11.256811 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 10002
I20260812 11:13:31.271394 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:42965
--webserver_interface=127.17.44.254
--webserver_port=38247
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:42965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:13:31.404500 10774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:31.404814 10774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:31.404892 10774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:31.409149 10774 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 11:13:31.409225 10774 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 11:13:31.409258 10774 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 11:13:31.409298 10774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:13:31.409344 10774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:31.409411 10774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:31.409456 10774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:13:31.409495 10774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:13:31.416693 10774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:42965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:42965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=38247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:31.418201 10774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:31.419598 10774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:31.429457 10785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:31.429436 10774 server_base.cc:1034] running on GCE node
W20260812 11:13:31.429461 10782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:31.429428 10781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:31.430327 10774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:31.431806 10774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:31.433012 10774 hybrid_clock.cc:648] HybridClock initialized: now 1786533211432976 us; error 94 us; skew 500 ppm
Aug 12 11:13:31 dist-test-slave-drl0 krb5kdc[9169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533211, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:13:31.439723 10774 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:13:31.440619 10774 webserver.cc:460] Webserver started at http://127.17.44.254:38247/ using document root <none> and password file <none>
I20260812 11:13:31.441249 10774 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:31.441346 10774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:31.445194 10774 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:13:31.447536 10797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:31.448868 10774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:13:31.449025 10774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "e11a4e54ab5e486bbc1162221cdcc66e"
format_stamp: "Formatted at 2026-08-12 11:12:48 on dist-test-slave-drl0"
server_key: "d9537b1b73f212dab36936e68f774fb8"
server_key_iv: "f2e95ccf4af2aaa2a51d4dddb7bbe705"
server_key_version: "encryptionkey@0"
I20260812 11:13:31.449564 10774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:31.479606 10774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:31.483006 10774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:31.483227 10774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:31.491772 10774 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:42965
I20260812 11:13:31.492221 10881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:42965 every 8 connection(s)
I20260812 11:13:31.493196 10774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:13:31.497982 10882 sys_catalog.cc:263] Verifying existing consensus state
I20260812 11:13:31.499809 10882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e: Bootstrap starting.
I20260812 11:13:31.501950 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 10774
I20260812 11:13:31.502430 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:46467
--local_ip_for_outbound_sockets=127.17.44.193
--tserver_master_addrs=127.17.44.254:42965
--webserver_port=40445
--webserver_interface=127.17.44.193
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:13:31.521723 10882 log.cc:824] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:31.533905 10882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 11:13:31.534353 10882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e: Bootstrap complete.
I20260812 11:13:31.539928 10882 raft_consensus.cc:348] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11a4e54ab5e486bbc1162221cdcc66e" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42965 } }
I20260812 11:13:31.540436 10882 raft_consensus.cc:729] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e11a4e54ab5e486bbc1162221cdcc66e, State: Initialized, Role: FOLLOWER
I20260812 11:13:31.541236 10882 consensus_queue.cc:260] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11a4e54ab5e486bbc1162221cdcc66e" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42965 } }
I20260812 11:13:31.541494 10882 raft_consensus.cc:388] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:13:31.541584 10882 raft_consensus.cc:482] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:13:31.541738 10882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 2 FOLLOWER]: Advancing to term 3
I20260812 11:13:31.543983 10882 raft_consensus.cc:504] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11a4e54ab5e486bbc1162221cdcc66e" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42965 } }
I20260812 11:13:31.544380 10882 leader_election.cc:302] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e11a4e54ab5e486bbc1162221cdcc66e; no voters:
I20260812 11:13:31.544764 10882 leader_election.cc:288] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 11:13:31.544893 10887 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 3 FOLLOWER]: Leader election won for term 3
I20260812 11:13:31.545138 10887 raft_consensus.cc:686] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [term 3 LEADER]: Becoming Leader. State: Replica: e11a4e54ab5e486bbc1162221cdcc66e, State: Running, Role: LEADER
I20260812 11:13:31.545472 10887 consensus_queue.cc:237] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11a4e54ab5e486bbc1162221cdcc66e" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42965 } }
I20260812 11:13:31.546087 10882 sys_catalog.cc:564] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:13:31.547039 10888 sys_catalog.cc:455] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e11a4e54ab5e486bbc1162221cdcc66e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11a4e54ab5e486bbc1162221cdcc66e" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42965 } } }
I20260812 11:13:31.547192 10888 sys_catalog.cc:458] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [sys.catalog]: This master's current role is: LEADER
I20260812 11:13:31.547477 10889 sys_catalog.cc:455] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e11a4e54ab5e486bbc1162221cdcc66e. Latest consensus state: current_term: 3 leader_uuid: "e11a4e54ab5e486bbc1162221cdcc66e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11a4e54ab5e486bbc1162221cdcc66e" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 42965 } } }
I20260812 11:13:31.547566 10889 sys_catalog.cc:458] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e [sys.catalog]: This master's current role is: LEADER
I20260812 11:13:31.548928 10899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:13:31.554315 10899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:13:31.555032 10899 catalog_manager.cc:1259] Loaded cluster ID: 40fef90bb8a04147982b4aeade7fda17
I20260812 11:13:31.555094 10899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:13:31.557624 10899 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:13:31.558398 10899 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e: Loaded TSK: 0
I20260812 11:13:31.558483 10899 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e11a4e54ab5e486bbc1162221cdcc66e: Loaded TSK: 1
F20260812 11:13:31.559288 10899 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb65a96f9ad google::LogMessage::Fail() at ??:0
@ 0x7fb65a97288a google::LogMessage::SendToLog() at ??:0
@ 0x7fb65a96f4de google::LogMessage::Flush() at ??:0
@ 0x7fb65a970a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb65d63318d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fb65d62bf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fb6652d47bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fb6652ae99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fb6652af178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7fb6652f0df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fb66530f82b std::function<>::operator()() at ??:0
@ 0x7fb6652aeb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fb6652aff3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb6652ac579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fb6652f0640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fb65cbc13d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb65cbc1c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fb65cbc33f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fb65cbb2512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb65e1f66db start_thread at ??:0
@ 0x7fb65946d61f clone at ??:0
W20260812 11:13:31.670569 10884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:31.670838 10884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:31.670885 10884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:31.675360 10884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:31.675491 10884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:31.675604 10884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:13:31.683032 10884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:46467
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=40445
--tserver_master_addrs=127.17.44.254:42965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:31.684203 10884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:31.685338 10884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:31.696444 10943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:31.696518 10884 server_base.cc:1034] running on GCE node
W20260812 11:13:31.696447 10946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:31.696445 10952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:31.697191 10884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:31.697976 10884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:31.699174 10884 hybrid_clock.cc:648] HybridClock initialized: now 1786533211699138 us; error 48 us; skew 500 ppm
Aug 12 11:13:31 dist-test-slave-drl0 krb5kdc[9169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533211, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:13:31.702745 10884 init.cc:377] Logged in from keytab as kudu/127.17.44.193@KRBTEST.COM (short username kudu)
I20260812 11:13:31.703742 10884 webserver.cc:460] Webserver started at http://127.17.44.193:40445/ using document root <none> and password file <none>
I20260812 11:13:31.704327 10884 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:31.704419 10884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:31.708366 10884 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260812 11:13:31.711611 10961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:31.713301 10884 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 11:13:31.713413 10884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "3cb5d354b65e44a2b85e18650db4d504"
format_stamp: "Formatted at 2026-08-12 11:12:48 on dist-test-slave-drl0"
server_key: "62d8732f984fd8728b64073943ec4d84"
server_key_iv: "bbee7c5fb387d3f02c1d2da362f1d466"
server_key_version: "encryptionkey@0"
I20260812 11:13:31.714015 10884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:31.755864 10884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:31.759964 10884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:31.760288 10884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:31.761072 10884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:13:31.762439 10973 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 11:13:31.768317 10884 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 11:13:31.768415 10884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260812 11:13:31.768499 10884 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 11:13:31.770171 10884 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 11:13:31.770244 10884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 11:13:31.770547 10973 tablet_bootstrap.cc:492] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504: Bootstrap starting.
I20260812 11:13:31.783237 10884 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:46467
I20260812 11:13:31.784492 10884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:13:31.785969 11127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:46467 every 8 connection(s)
I20260812 11:13:31.791893 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 10884
I20260812 11:13:31.792342 10973 log.cc:824] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:31.792454 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:34241
--local_ip_for_outbound_sockets=127.17.44.194
--tserver_master_addrs=127.17.44.254:42965
--webserver_port=46293
--webserver_interface=127.17.44.194
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:13:31.799921 10973 tablet_bootstrap.cc:492] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 11:13:31.800312 10973 tablet_bootstrap.cc:492] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504: Bootstrap complete.
I20260812 11:13:31.801570 10973 ts_tablet_manager.cc:1397] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504: Time spent bootstrapping tablet: real 0.031s user 0.021s sys 0.007s
I20260812 11:13:31.804126 10973 raft_consensus.cc:348] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } } peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } }
I20260812 11:13:31.804632 10973 raft_consensus.cc:729] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cb5d354b65e44a2b85e18650db4d504, State: Initialized, Role: FOLLOWER
I20260812 11:13:31.805305 10973 consensus_queue.cc:260] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } } peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } }
I20260812 11:13:31.806447 10973 ts_tablet_manager.cc:1428] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260812 11:13:31.821934 11132 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:13:31.787732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.193:60359)
0812 11:13:31.788519 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:31.788579 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:31.789644 (+ 1065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:31.821026 (+ 31382us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":436,"spinlock_wait_cycles":127872,"thread_start_us":196,"threads_started":1}
W20260812 11:13:31.822490 11128 heartbeater.cc:644] Failed to heartbeat to 127.17.44.254:42965 (0 consecutive failures): Network error: Failed to ping master at 127.17.44.254:42965: Client connection negotiation failed: client connection to 127.17.44.254:42965: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 11:13:31.823323 11132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:31.823068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.193:58211)
0812 11:13:31.823194 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:31.823269 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 11:13:31.824095 11132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:31.823775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.193:48921)
0812 11:13:31.823971 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:31.824040 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260812 11:13:31.824323 11128 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 11:13:31.932585 11134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:31.932850 11134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:31.932895 11134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:31.936992 11134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:31.937211 11134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:31.937662 11134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:13:31.944118 11134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:34241
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=46293
--tserver_master_addrs=127.17.44.254:42965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:31.945331 11134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:31.946447 11134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:31.956651 11149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:31.956624 11151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:31.956876 11134 server_base.cc:1034] running on GCE node
W20260812 11:13:31.956960 11148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:31.957362 11134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:31.957984 11134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:31.959214 11134 hybrid_clock.cc:648] HybridClock initialized: now 1786533211959191 us; error 33 us; skew 500 ppm
Aug 12 11:13:31 dist-test-slave-drl0 krb5kdc[9169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533211, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:13:31.962239 11134 init.cc:377] Logged in from keytab as kudu/127.17.44.194@KRBTEST.COM (short username kudu)
I20260812 11:13:31.963222 11134 webserver.cc:460] Webserver started at http://127.17.44.194:46293/ using document root <none> and password file <none>
I20260812 11:13:31.963801 11134 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:31.963905 11134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:31.968058 11134 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260812 11:13:31.970278 11165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:31.971642 11134 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:13:31.971786 11134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "bb17c21d0ed14f238506c985e17099da"
format_stamp: "Formatted at 2026-08-12 11:12:48 on dist-test-slave-drl0"
server_key: "9827b9bd2d0cfec07cff7cf005a35042"
server_key_iv: "c4e7559908d10abef0b580a987ccd072"
server_key_version: "encryptionkey@0"
I20260812 11:13:31.972338 11134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:31.988791 11134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:31.992559 11134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:31.992790 11134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:31.993413 11134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:13:31.994297 11134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:13:31.994385 11134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:31.994465 11134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:13:31.994514 11134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:32.006150 11134 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:34241
I20260812 11:13:32.006181 11334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:34241 every 8 connection(s)
I20260812 11:13:32.007165 11134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:13:32.011292 11341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:32.009584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.194:48447)
0812 11:13:32.010358 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:32.010470 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":150,"threads_started":1}
W20260812 11:13:32.011842 11335 heartbeater.cc:644] Failed to heartbeat to 127.17.44.254:42965 (0 consecutive failures): Network error: Failed to ping master at 127.17.44.254:42965: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
I20260812 11:13:32.012729 11341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:32.012293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.194:59233)
0812 11:13:32.012428 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:32.012583 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 11:13:32.013448 11341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:32.013222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.194:42661)
0812 11:13:32.013338 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:32.013396 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 11:13:32.013629 11335 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 11:13:32.015846 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 11134
I20260812 11:13:32.016331 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:43595
--local_ip_for_outbound_sockets=127.17.44.195
--tserver_master_addrs=127.17.44.254:42965
--webserver_port=34383
--webserver_interface=127.17.44.195
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:38477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:13:32.135820 11342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:32.136070 11342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:32.136111 11342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:32.140214 11342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:32.140266 11342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:32.140326 11342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:13:32.145774 11342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:38477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:43595
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=34383
--tserver_master_addrs=127.17.44.254:42965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:32.147027 11342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:32.148090 11342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:32.158511 11347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:32.158520 11349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:32.158653 11342 server_base.cc:1034] running on GCE node
W20260812 11:13:32.158520 11351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:32.159509 11342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:32.160174 11342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:32.161549 11342 hybrid_clock.cc:648] HybridClock initialized: now 1786533212161503 us; error 33 us; skew 500 ppm
Aug 12 11:13:32 dist-test-slave-drl0 krb5kdc[9169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533212, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:13:32.165972 11342 init.cc:377] Logged in from keytab as kudu/127.17.44.195@KRBTEST.COM (short username kudu)
I20260812 11:13:32.167002 11342 webserver.cc:460] Webserver started at http://127.17.44.195:34383/ using document root <none> and password file <none>
I20260812 11:13:32.167631 11342 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:32.167738 11342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:32.172116 11342 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:13:32.174913 11365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:32.176581 11342 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 11:13:32.176712 11342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "acba0798196d48d691a96b28eaed9b11"
format_stamp: "Formatted at 2026-08-12 11:12:49 on dist-test-slave-drl0"
server_key: "f7883bfd4ed9853bab4033de0aaeca3b"
server_key_iv: "498fb7c6b4df3389e041801c76ac2a35"
server_key_version: "encryptionkey@0"
I20260812 11:13:32.177269 11342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:32.199380 11342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:32.202796 11342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:32.203038 11342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:32.203603 11342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:13:32.204411 11342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:13:32.204460 11342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:32.204500 11342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:13:32.204515 11342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:32.214627 11342 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:43595
I20260812 11:13:32.214677 11520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:43595 every 8 connection(s)
I20260812 11:13:32.215735 11342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:13:32.216231 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 11342
I20260812 11:13:32.220237 11527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:32.218756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.195:54825)
0812 11:13:32.219445 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:32.219545 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":129,"threads_started":1}
W20260812 11:13:32.220767 11522 heartbeater.cc:644] Failed to heartbeat to 127.17.44.254:42965 (0 consecutive failures): Network error: Failed to ping master at 127.17.44.254:42965: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
I20260812 11:13:32.221475 11527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:32.221228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.195:47199)
0812 11:13:32.221363 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:32.221417 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 11:13:32.222568 11527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:32.222231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.195:53483)
0812 11:13:32.222414 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:32.222475 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260812 11:13:32.223148 11522 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 11:13:32.825536 11530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:32.825012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.193:39339)
0812 11:13:32.825336 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:32.825414 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":81,"threads_started":1}
I20260812 11:13:33.015215 11533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:33.014379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.194:34553)
0812 11:13:33.014828 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:33.014908 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":113,"threads_started":1}
I20260812 11:13:33.224347 11534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:33.223854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.195:48987)
0812 11:13:33.224172 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:33.224237 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":92,"threads_started":1}
I20260812 11:13:33.360280 11536 raft_consensus.cc:482] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:13:33.360565 11536 raft_consensus.cc:504] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } } peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } }
I20260812 11:13:33.361701 11536 leader_election.cc:288] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bb17c21d0ed14f238506c985e17099da (127.17.44.194:34241), acba0798196d48d691a96b28eaed9b11 (127.17.44.195:43595)
Aug 12 11:13:33 dist-test-slave-drl0 krb5kdc[9169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533211, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.194@KRBTEST.COM
Aug 12 11:13:33 dist-test-slave-drl0 krb5kdc[9169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533211, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.195@KRBTEST.COM
I20260812 11:13:33.381155 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:33.361970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:43595 (local address 127.17.44.193:48669)
0812 11:13:33.362400 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:33.362414 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:33.362476 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:33.364630 (+ 2154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:33.364637 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:33.365250 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:13:33.366082 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:33.366105 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:33.368101 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:33.368105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:33.369780 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:33.369793 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:33.370773 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:33.370778 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:33.371176 (+ 398us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:33.372051 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:13:33.372079 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:13:33.374584 (+ 2505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:13:33.377245 (+ 2661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:33.377251 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:33.377264 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:33.377866 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:33.378705 (+ 839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:33.378708 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:33.378710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:33.378961 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:33.379888 (+ 927us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:13:33.379895 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:13:33.380291 (+ 396us) client_negotiation.cc:769] Sending connection context
0812 11:13:33.380516 (+ 225us) client_negotiation.cc:240] Negotiation successful
0812 11:13:33.380916 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":532,"spinlock_wait_cycles":38912,"thread_start_us":64,"threads_started":1}
I20260812 11:13:33.381189 11541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:33.361970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:34241 (local address 127.17.44.193:60015)
0812 11:13:33.362320 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:33.362336 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:33.362467 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:33.364630 (+ 2163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:33.364637 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:33.365250 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:13:33.366082 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:33.366105 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:33.368101 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:33.368105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:33.369782 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:33.369793 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:33.370773 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:33.370778 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:33.371177 (+ 399us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:33.372051 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:13:33.372076 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:13:33.374131 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:13:33.377402 (+ 3271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:33.377405 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:33.377407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:33.377866 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:33.378693 (+ 827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:33.378697 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:33.378705 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:33.378961 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:33.379888 (+ 927us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:13:33.379895 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:13:33.380291 (+ 396us) client_negotiation.cc:769] Sending connection context
0812 11:13:33.380516 (+ 225us) client_negotiation.cc:240] Negotiation successful
0812 11:13:33.380987 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":682,"spinlock_wait_cycles":60416,"thread_start_us":84,"threads_started":1}
I20260812 11:13:33.381196 11546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:33.362224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:48669 (local address 127.17.44.195:43595)
0812 11:13:33.362656 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:33.362685 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:33.362730 (+ 45us) server_negotiation.cc:407] Connection header received
0812 11:13:33.363683 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:33.363692 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:33.364139 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:33.364483 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:33.366276 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:33.367912 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:33.369935 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:33.370619 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:33.370791 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:33.374864 (+ 4073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:33.374870 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:33.374880 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:13:33.374907 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:33.377135 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:33.378142 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:33.378154 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:33.378165 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:33.378490 (+ 325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:33.379101 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:33.379106 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:33.379108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:33.379692 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:33.379852 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:33.380692 (+ 840us) server_negotiation.cc:299] Negotiation successful
0812 11:13:33.381009 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":112,"threads_started":1}
I20260812 11:13:33.381196 11544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:33.362226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:60015 (local address 127.17.44.194:34241)
0812 11:13:33.362656 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:33.362685 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:33.362730 (+ 45us) server_negotiation.cc:407] Connection header received
0812 11:13:33.363683 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:33.363692 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:33.364139 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:33.364483 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:33.366432 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:33.367912 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:33.369931 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:33.370619 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:33.370791 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:33.374411 (+ 3620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:33.374418 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:33.374434 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:13:33.374476 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:33.377278 (+ 2802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:33.378143 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:33.378154 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:33.378165 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:33.378491 (+ 326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:33.379101 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:33.379106 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:33.379108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:33.379692 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:33.379852 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:33.380706 (+ 854us) server_negotiation.cc:299] Negotiation successful
0812 11:13:33.381009 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":97,"threads_started":1}
W20260812 11:13:33.383985 10970 leader_election.cc:341] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bb17c21d0ed14f238506c985e17099da (127.17.44.194:34241): Not found: Tablet not found: f32e1c829fc0424ab9c3e539391ad080
W20260812 11:13:33.384268 10966 leader_election.cc:341] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer acba0798196d48d691a96b28eaed9b11 (127.17.44.195:43595): Not found: Tablet not found: f32e1c829fc0424ab9c3e539391ad080
I20260812 11:13:33.384443 10966 leader_election.cc:302] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3cb5d354b65e44a2b85e18650db4d504; no voters: acba0798196d48d691a96b28eaed9b11, bb17c21d0ed14f238506c985e17099da
I20260812 11:13:33.384722 11536 raft_consensus.cc:2726] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 11:13:33.826818 11541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:33.826505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.193:41767)
0812 11:13:33.826661 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:33.826731 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 11:13:34.016556 11547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:34.016119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.194:39825)
0812 11:13:34.016402 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:34.016464 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":78,"threads_started":1}
I20260812 11:13:34.225687 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:34.225211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.195:58737)
0812 11:13:34.225521 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:34.225582 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":95,"threads_started":1}
I20260812 11:13:34.828158 11550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:34.827712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.193:39385)
0812 11:13:34.828002 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:34.828058 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":87,"threads_started":1}
I20260812 11:13:35.018090 11555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:35.017627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.194:54915)
0812 11:13:35.017922 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:35.017981 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":80,"threads_started":1}
I20260812 11:13:35.227182 11557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:35.226603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.195:34853)
0812 11:13:35.226884 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:35.227072 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":97,"threads_started":1}
I20260812 11:13:35.435292 11560 raft_consensus.cc:482] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:13:35.435515 11560 raft_consensus.cc:504] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } } peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } }
I20260812 11:13:35.436537 11560 leader_election.cc:288] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bb17c21d0ed14f238506c985e17099da (127.17.44.194:34241), acba0798196d48d691a96b28eaed9b11 (127.17.44.195:43595)
W20260812 11:13:35.437077 10970 leader_election.cc:341] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bb17c21d0ed14f238506c985e17099da (127.17.44.194:34241): Not found: Tablet not found: f32e1c829fc0424ab9c3e539391ad080
W20260812 11:13:35.437213 10966 leader_election.cc:341] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer acba0798196d48d691a96b28eaed9b11 (127.17.44.195:43595): Not found: Tablet not found: f32e1c829fc0424ab9c3e539391ad080
I20260812 11:13:35.437335 10966 leader_election.cc:302] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3cb5d354b65e44a2b85e18650db4d504; no voters: acba0798196d48d691a96b28eaed9b11, bb17c21d0ed14f238506c985e17099da
I20260812 11:13:35.437508 11560 raft_consensus.cc:2726] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 11:13:35.829504 11562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:35.829028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.193:33815)
0812 11:13:35.829331 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:35.829387 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":88,"threads_started":1}
I20260812 11:13:36.019742 11564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:36.019300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.194:59541)
0812 11:13:36.019585 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:36.019652 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":90,"threads_started":1}
I20260812 11:13:36.228643 11565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:36.228217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.195:54911)
0812 11:13:36.228491 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:36.228547 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":84,"threads_started":1}
I20260812 11:13:36.831040 11566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:36.830437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.193:38077)
0812 11:13:36.830784 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:36.830854 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":145,"threads_started":1}
I20260812 11:13:37.020923 11567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:37.020480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.194:48511)
0812 11:13:37.020778 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:37.020834 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":77,"threads_started":1}
I20260812 11:13:37.229913 11568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:37.229484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.195:48675)
0812 11:13:37.229768 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:37.229822 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":100,"threads_started":1}
I20260812 11:13:37.673974 11569 raft_consensus.cc:482] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:13:37.674118 11569 raft_consensus.cc:504] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } } peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } }
I20260812 11:13:37.674566 11569 leader_election.cc:288] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bb17c21d0ed14f238506c985e17099da (127.17.44.194:34241), acba0798196d48d691a96b28eaed9b11 (127.17.44.195:43595)
W20260812 11:13:37.675668 10966 leader_election.cc:341] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer acba0798196d48d691a96b28eaed9b11 (127.17.44.195:43595): Not found: Tablet not found: f32e1c829fc0424ab9c3e539391ad080
W20260812 11:13:37.675830 10970 leader_election.cc:341] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bb17c21d0ed14f238506c985e17099da (127.17.44.194:34241): Not found: Tablet not found: f32e1c829fc0424ab9c3e539391ad080
I20260812 11:13:37.675951 10970 leader_election.cc:302] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3cb5d354b65e44a2b85e18650db4d504; no voters: acba0798196d48d691a96b28eaed9b11, bb17c21d0ed14f238506c985e17099da
I20260812 11:13:37.676239 11569 raft_consensus.cc:2726] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 11:13:37.832404 11570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:37.831821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.193:44609)
0812 11:13:37.832170 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:37.832224 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":86,"threads_started":1}
I20260812 11:13:38.022293 11572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:38.021847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.194:35985)
0812 11:13:38.022140 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:38.022200 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":96,"threads_started":1}
I20260812 11:13:38.231474 11573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:38.231004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.195:38929)
0812 11:13:38.231294 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:38.231353 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":78,"threads_started":1}
I20260812 11:13:38.834400 11575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:38.833894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.193:40893)
0812 11:13:38.834229 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:38.834292 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":111,"threads_started":1}
I20260812 11:13:39.024135 11578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:39.023511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.194:35223)
0812 11:13:39.023968 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:39.024034 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":118,"threads_started":1}
I20260812 11:13:39.232903 11579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:39.232370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.195:52779)
0812 11:13:39.232744 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:39.232807 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":84,"threads_started":1}
I20260812 11:13:39.836417 11581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:39.835811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.193:60351)
0812 11:13:39.836187 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:39.836278 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":124,"threads_started":1}
I20260812 11:13:40.025475 11583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:40.025053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.194:58539)
0812 11:13:40.025325 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:40.025381 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":80,"threads_started":1}
I20260812 11:13:40.234719 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:40.234002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.195:47921)
0812 11:13:40.234491 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:40.234590 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":94,"threads_started":1}
I20260812 11:13:40.257908 11585 raft_consensus.cc:482] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:13:40.258069 11585 raft_consensus.cc:504] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } } peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } }
I20260812 11:13:40.258628 11585 leader_election.cc:288] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bb17c21d0ed14f238506c985e17099da (127.17.44.194:34241), acba0798196d48d691a96b28eaed9b11 (127.17.44.195:43595)
W20260812 11:13:40.259634 10970 leader_election.cc:341] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bb17c21d0ed14f238506c985e17099da (127.17.44.194:34241): Not found: Tablet not found: f32e1c829fc0424ab9c3e539391ad080
W20260812 11:13:40.259739 10966 leader_election.cc:341] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer acba0798196d48d691a96b28eaed9b11 (127.17.44.195:43595): Not found: Tablet not found: f32e1c829fc0424ab9c3e539391ad080
I20260812 11:13:40.259903 10966 leader_election.cc:302] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3cb5d354b65e44a2b85e18650db4d504; no voters: acba0798196d48d691a96b28eaed9b11, bb17c21d0ed14f238506c985e17099da
I20260812 11:13:40.260105 11585 raft_consensus.cc:2726] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 11:13:40.838223 11587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:40.837561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.193:54575)
0812 11:13:40.837997 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:40.838080 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":139,"threads_started":1}
I20260812 11:13:41.026715 11588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:41.026263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.194:60591)
0812 11:13:41.026556 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:41.026624 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":110,"threads_started":1}
I20260812 11:13:41.236300 11589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:41.235701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.195:56947)
0812 11:13:41.235998 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:41.236196 (+ 198us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":100,"threads_started":1}
I20260812 11:13:41.840068 11590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:41.839597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.193:53121)
0812 11:13:41.839890 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:41.839960 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":92,"threads_started":1}
I20260812 11:13:42.027969 11591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:42.027517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.194:34371)
0812 11:13:42.027806 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:42.027873 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":77,"threads_started":1}
I20260812 11:13:42.238018 11593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:42.237270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.195:56823)
0812 11:13:42.237779 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:42.237835 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":92,"threads_started":1}
I20260812 11:13:42.841364 11595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:42.840924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.193:49327)
0812 11:13:42.841209 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:42.841264 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":79,"threads_started":1}
I20260812 11:13:43.029388 11596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:43.028879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.194:55379)
0812 11:13:43.029224 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:43.029284 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":89,"threads_started":1}
I20260812 11:13:43.239814 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:43.239262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.195:58135)
0812 11:13:43.239650 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:43.239705 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":150,"threads_started":1}
I20260812 11:13:43.843096 11598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:43.842483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.193:35879)
0812 11:13:43.842840 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:43.842909 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":113,"threads_started":1}
I20260812 11:13:44.032289 11599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:44.031557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.194:46105)
0812 11:13:44.032073 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:44.032154 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":205,"threads_started":1}
I20260812 11:13:44.241524 11601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:44.240888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.195:51821)
0812 11:13:44.241186 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:44.241245 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":124,"threads_started":1}
I20260812 11:13:44.844417 11602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:44.843951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.193:47987)
0812 11:13:44.844255 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:44.844307 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":86,"threads_started":1}
I20260812 11:13:44.908384 11604 raft_consensus.cc:482] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:13:44.908536 11604 raft_consensus.cc:504] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb17c21d0ed14f238506c985e17099da" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34241 } } peers { permanent_uuid: "acba0798196d48d691a96b28eaed9b11" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 43595 } } peers { permanent_uuid: "3cb5d354b65e44a2b85e18650db4d504" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 46467 } }
I20260812 11:13:44.908918 11604 leader_election.cc:288] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bb17c21d0ed14f238506c985e17099da (127.17.44.194:34241), acba0798196d48d691a96b28eaed9b11 (127.17.44.195:43595)
W20260812 11:13:44.910017 10966 leader_election.cc:341] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer acba0798196d48d691a96b28eaed9b11 (127.17.44.195:43595): Not found: Tablet not found: f32e1c829fc0424ab9c3e539391ad080
W20260812 11:13:44.910120 10970 leader_election.cc:341] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bb17c21d0ed14f238506c985e17099da (127.17.44.194:34241): Not found: Tablet not found: f32e1c829fc0424ab9c3e539391ad080
I20260812 11:13:44.910175 10970 leader_election.cc:302] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3cb5d354b65e44a2b85e18650db4d504; no voters: acba0798196d48d691a96b28eaed9b11, bb17c21d0ed14f238506c985e17099da
I20260812 11:13:44.910328 11604 raft_consensus.cc:2726] T f32e1c829fc0424ab9c3e539391ad080 P 3cb5d354b65e44a2b85e18650db4d504 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 11:13:45.034461 11608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:45.033466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.194:58393)
0812 11:13:45.033913 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:45.034326 (+ 413us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":127,"threads_started":1}
I20260812 11:13:45.243319 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:45.242762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.195:46965)
0812 11:13:45.243138 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:45.243200 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":311,"threads_started":1}
I20260812 11:13:45.845717 11611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:45.845286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.193:54159)
0812 11:13:45.845564 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:45.845618 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":114,"threads_started":1}
I20260812 11:13:46.035934 11612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:46.035442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.194:41921)
0812 11:13:46.035757 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:46.035821 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":78,"threads_started":1}
I20260812 11:13:46.244894 11615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:46.244330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.195:51081)
0812 11:13:46.244716 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:46.244782 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":121,"threads_started":1}
I20260812 11:13:46.847348 11616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:46.846830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.193:56045)
0812 11:13:46.847178 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:46.847240 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":145,"threads_started":1}
I20260812 11:13:47.037352 11617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:47.036918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.194:52123)
0812 11:13:47.037196 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:47.037250 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":77,"threads_started":1}
I20260812 11:13:47.223402 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 10884
I20260812 11:13:47.237423 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 11134
I20260812 11:13:47.246582 11618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:47.246029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:42965 (local address 127.17.44.195:60817)
0812 11:13:47.246351 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:47.246431 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.254:42965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":73,"threads_started":1}
I20260812 11:13:47.249212 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 11342
2026-08-12T11:13:47Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (60440 ms)
[----------] 30 tests from SecurityITest (265516 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:13:47 dist-test-slave-drl0 krb5kdc[11623](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:13:47 dist-test-slave-drl0 krb5kdc[11623](info): set up 2 sockets
Aug 12 11:13:47 dist-test-slave-drl0 krb5kdc[11623](info): commencing operation
krb5kdc: starting...
W20260812 11:13:48.558156 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:13:48 dist-test-slave-drl0 krb5kdc[11623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:13:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:13:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:13:48.702808 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:46559
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:44683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:46559
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:13:48.819650 11638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:48.819942 11638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:48.820016 11638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:48.824362 11638 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:13:48.824465 11638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:48.824506 11638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:48.824662 11638 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:13:48.824747 11638 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:13:48.830570 11638 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:44683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:46559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:46559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:48.831881 11638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:48.832882 11638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:48.842247 11649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:48.842242 11645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:48.842293 11647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:48.842655 11638 server_base.cc:1034] running on GCE node
I20260812 11:13:48.843495 11638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:48.844535 11638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:48.845764 11638 hybrid_clock.cc:648] HybridClock initialized: now 1786533228845684 us; error 78 us; skew 500 ppm
Aug 12 11:13:48 dist-test-slave-drl0 krb5kdc[11623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533228, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:13:48.849162 11638 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:13:48.850103 11638 webserver.cc:460] Webserver started at http://127.17.44.254:43269/ using document root <none> and password file <none>
I20260812 11:13:48.850687 11638 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:48.850755 11638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:48.851033 11638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:13:48.853236 11638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "38d7bcdf74ee4d25890fb35ae561e6f1"
format_stamp: "Formatted at 2026-08-12 11:13:48 on dist-test-slave-drl0"
server_key: "53cd981af9fa90795b3796ecfef916a8"
server_key_iv: "1574347b544d74471de4e27e09eac5f7"
server_key_version: "encryptionkey@0"
I20260812 11:13:48.853837 11638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "38d7bcdf74ee4d25890fb35ae561e6f1"
format_stamp: "Formatted at 2026-08-12 11:13:48 on dist-test-slave-drl0"
server_key: "53cd981af9fa90795b3796ecfef916a8"
server_key_iv: "1574347b544d74471de4e27e09eac5f7"
server_key_version: "encryptionkey@0"
I20260812 11:13:48.857645 11638 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 11:13:48.860801 11657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:48.862242 11638 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:13:48.862406 11638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "38d7bcdf74ee4d25890fb35ae561e6f1"
format_stamp: "Formatted at 2026-08-12 11:13:48 on dist-test-slave-drl0"
server_key: "53cd981af9fa90795b3796ecfef916a8"
server_key_iv: "1574347b544d74471de4e27e09eac5f7"
server_key_version: "encryptionkey@0"
I20260812 11:13:48.862526 11638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:48.878494 11638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:48.881983 11638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:48.882300 11638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:48.890148 11638 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:46559
I20260812 11:13:48.890177 11736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:46559 every 8 connection(s)
I20260812 11:13:48.891124 11638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:13:48.893880 11737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:48.900499 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 11638
I20260812 11:13:48.900671 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:13:48.900990 17587 external_mini_cluster.cc:1428] Setting key 79e7b230d3d0ba53711dbcc6d4d33c82
I20260812 11:13:48.901372 11737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1: Bootstrap starting.
I20260812 11:13:48.904681 11737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:48.905740 11737 log.cc:824] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:48.908169 11737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1: No bootstrap required, opened a new log
Aug 12 11:13:48 dist-test-slave-drl0 krb5kdc[11623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:13:48.912977 11737 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d7bcdf74ee4d25890fb35ae561e6f1" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 46559 } }
I20260812 11:13:48.913231 11737 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:48.913326 11737 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38d7bcdf74ee4d25890fb35ae561e6f1, State: Initialized, Role: FOLLOWER
I20260812 11:13:48.914585 11737 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d7bcdf74ee4d25890fb35ae561e6f1" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 46559 } }
I20260812 11:13:48.914897 11737 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:13:48.915023 11737 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:13:48.915169 11737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:48.916723 11737 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d7bcdf74ee4d25890fb35ae561e6f1" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 46559 } }
I20260812 11:13:48.917212 11737 leader_election.cc:302] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38d7bcdf74ee4d25890fb35ae561e6f1; no voters:
I20260812 11:13:48.917665 11737 leader_election.cc:288] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:13:48.917760 11745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:13:48.918196 11745 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1 [term 1 LEADER]: Becoming Leader. State: Replica: 38d7bcdf74ee4d25890fb35ae561e6f1, State: Running, Role: LEADER
I20260812 11:13:48.918645 11745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d7bcdf74ee4d25890fb35ae561e6f1" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 46559 } }
I20260812 11:13:48.919242 11737 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:13:48.919646 11742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:48.902814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34882 (local address 127.17.44.254:46559)
0812 11:13:48.903508 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:48.903523 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:48.903561 (+ 38us) server_negotiation.cc:407] Connection header received
0812 11:13:48.904305 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:48.904322 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:48.904645 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:48.905003 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:48.905971 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:48.907355 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:48.908553 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:48.909301 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:48.909533 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:48.912586 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:48.912591 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:48.912604 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:13:48.912629 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:48.915957 (+ 3328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:48.916577 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:48.916586 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:48.916595 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:48.916770 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:48.917144 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:48.917154 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:48.917155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:48.917771 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:48.917984 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:48.918327 (+ 343us) server_negotiation.cc:299] Negotiation successful
0812 11:13:48.918668 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":192,"threads_started":1}
I20260812 11:13:48.920094 11745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38d7bcdf74ee4d25890fb35ae561e6f1. Latest consensus state: current_term: 1 leader_uuid: "38d7bcdf74ee4d25890fb35ae561e6f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d7bcdf74ee4d25890fb35ae561e6f1" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 46559 } } }
I20260812 11:13:48.920545 11750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38d7bcdf74ee4d25890fb35ae561e6f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d7bcdf74ee4d25890fb35ae561e6f1" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 46559 } } }
I20260812 11:13:48.920609 11745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1 [sys.catalog]: This master's current role is: LEADER
I20260812 11:13:48.920979 11750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1 [sys.catalog]: This master's current role is: LEADER
W20260812 11:13:48.925567 11770 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:13:48.925637 11770 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:13:48.925717 11760 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:13:48.926406 11760 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:13:48.934546 11760 catalog_manager.cc:1347] Generated new cluster ID: e9f6b60e86654d0980178d05cee8adda
I20260812 11:13:48.934620 11760 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:13:48.941207 11760 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:13:48.942368 11760 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:13:48.952095 11760 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1: Generated new TSK 0
I20260812 11:13:48.952847 11760 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
I20260812 11:13:48.987098 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:46559
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:44683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:13:49.113333 11778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:49.113636 11778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:49.113718 11778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:49.117865 11778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:49.117959 11778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:49.118046 11778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:13:49.123564 11778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:44683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:46559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:49.124614 11778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:49.125998 11778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:49.136266 11789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:49.136274 11793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:49.136286 11790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:49.136540 11778 server_base.cc:1034] running on GCE node
I20260812 11:13:49.136909 11778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:49.137526 11778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:49.138810 11778 hybrid_clock.cc:648] HybridClock initialized: now 1786533229138735 us; error 70 us; skew 500 ppm
Aug 12 11:13:49 dist-test-slave-drl0 krb5kdc[11623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533229, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:13:49.142619 11778 init.cc:377] Logged in from keytab as kudu/127.17.44.193@KRBTEST.COM (short username kudu)
I20260812 11:13:49.143718 11778 webserver.cc:460] Webserver started at http://127.17.44.193:46807/ using document root <none> and password file <none>
I20260812 11:13:49.144273 11778 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:49.144336 11778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:49.144511 11778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:13:49.146844 11778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "d864eb97813c41e3ac010aac6522335c"
format_stamp: "Formatted at 2026-08-12 11:13:49 on dist-test-slave-drl0"
server_key: "e2a5eb67343833166660e1402ee07191"
server_key_iv: "f8c4c4b7f5fd0048912f49690deccbed"
server_key_version: "encryptionkey@0"
I20260812 11:13:49.147567 11778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "d864eb97813c41e3ac010aac6522335c"
format_stamp: "Formatted at 2026-08-12 11:13:49 on dist-test-slave-drl0"
server_key: "e2a5eb67343833166660e1402ee07191"
server_key_iv: "f8c4c4b7f5fd0048912f49690deccbed"
server_key_version: "encryptionkey@0"
I20260812 11:13:49.151610 11778 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 11:13:49.154225 11800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:49.155670 11778 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:13:49.155808 11778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "d864eb97813c41e3ac010aac6522335c"
format_stamp: "Formatted at 2026-08-12 11:13:49 on dist-test-slave-drl0"
server_key: "e2a5eb67343833166660e1402ee07191"
server_key_iv: "f8c4c4b7f5fd0048912f49690deccbed"
server_key_version: "encryptionkey@0"
I20260812 11:13:49.155949 11778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:49.173096 11778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:49.176393 11778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:49.176591 11778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:49.177194 11778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:13:49.178021 11778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:13:49.178093 11778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:49.178159 11778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:13:49.178208 11778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:49.190053 11778 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:41619
I20260812 11:13:49.190081 11981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:41619 every 8 connection(s)
I20260812 11:13:49.190979 11778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:13:49.196564 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 11778
I20260812 11:13:49.196861 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:13:49.197201 17587 external_mini_cluster.cc:1428] Setting key c88fc14d1e12193c4c4acb6a04ca5bbb
Aug 12 11:13:49 dist-test-slave-drl0 krb5kdc[11623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533229, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:13:49.207481 11742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:49.192985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:33185 (local address 127.17.44.254:46559)
0812 11:13:49.193155 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:49.193158 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:49.194465 (+ 1307us) server_negotiation.cc:407] Connection header received
0812 11:13:49.195458 (+ 993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:49.195462 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:49.195522 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:49.195634 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:49.196944 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.197594 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.199296 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.199526 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.199621 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:49.202506 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:49.202512 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:49.202518 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:13:49.202551 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:49.204765 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:49.205308 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:49.205315 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:49.205319 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:49.205430 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:49.206104 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:49.206109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:49.206113 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:49.206410 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:49.206529 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:49.207127 (+ 598us) server_negotiation.cc:299] Negotiation successful
0812 11:13:49.207277 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 11:13:49.208765 11985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:49.193367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:46559 (local address 127.17.44.193:33185)
0812 11:13:49.194307 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:49.194349 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:49.195255 (+ 906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:49.195801 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:49.195807 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:49.196147 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:13:49.196610 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:49.196651 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.197802 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.197806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:49.199124 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:49.199138 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.199694 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.199701 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:49.199949 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:49.200599 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:13:49.200625 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:13:49.202335 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:13:49.204914 (+ 2579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:49.204919 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:49.204930 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:49.205188 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:49.205574 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:49.205577 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:49.205578 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:49.205986 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:49.206570 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:13:49.206575 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:13:49.206817 (+ 242us) client_negotiation.cc:769] Sending connection context
0812 11:13:49.207062 (+ 245us) client_negotiation.cc:240] Negotiation successful
0812 11:13:49.207345 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"thread_start_us":563,"threads_started":1}
WARNING: no policy specified for kudu/127.17.44.194@KRBTEST.COM; defaulting to no policy
I20260812 11:13:49.210501 11982 heartbeater.cc:347] Connected to a master server at 127.17.44.254:46559
I20260812 11:13:49.210846 11982 heartbeater.cc:464] Registering TS with master...
I20260812 11:13:49.211429 11982 heartbeater.cc:511] Master 127.17.44.254:46559 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.194@KRBTEST.COM" created.
I20260812 11:13:49.213809 11686 ts_manager.cc:194] Registered new tserver with Master: d864eb97813c41e3ac010aac6522335c (127.17.44.193:41619)
I20260812 11:13:49.215478 11686 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.193@KRBTEST.COM'} at 127.17.44.193:33185
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
I20260812 11:13:49.231194 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:46559
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:44683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:13:49.354864 11989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:49.355304 11989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:49.355350 11989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:49.359668 11989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:49.359721 11989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:49.359800 11989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:13:49.366289 11989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:44683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:46559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:49.367551 11989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:49.368669 11989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:49.379086 11996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:49.379051 11998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:49.379074 11994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:49.379274 11989 server_base.cc:1034] running on GCE node
I20260812 11:13:49.379769 11989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:49.380364 11989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:49.381562 11989 hybrid_clock.cc:648] HybridClock initialized: now 1786533229381532 us; error 40 us; skew 500 ppm
Aug 12 11:13:49 dist-test-slave-drl0 krb5kdc[11623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533229, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:13:49.385207 11989 init.cc:377] Logged in from keytab as kudu/127.17.44.194@KRBTEST.COM (short username kudu)
I20260812 11:13:49.386101 11989 webserver.cc:460] Webserver started at http://127.17.44.194:45689/ using document root <none> and password file <none>
I20260812 11:13:49.386670 11989 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:49.386777 11989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:49.387187 11989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:13:49.389358 11989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "3a2c46698e2c44b691e866aabd1e05e4"
format_stamp: "Formatted at 2026-08-12 11:13:49 on dist-test-slave-drl0"
server_key: "05083fb7e765a519822ebfc37384ab56"
server_key_iv: "1a96d350f388056207e9a802e6ea70b1"
server_key_version: "encryptionkey@0"
I20260812 11:13:49.389938 11989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "3a2c46698e2c44b691e866aabd1e05e4"
format_stamp: "Formatted at 2026-08-12 11:13:49 on dist-test-slave-drl0"
server_key: "05083fb7e765a519822ebfc37384ab56"
server_key_iv: "1a96d350f388056207e9a802e6ea70b1"
server_key_version: "encryptionkey@0"
I20260812 11:13:49.394210 11989 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 11:13:49.397336 12018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:49.398718 11989 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:13:49.398854 11989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "3a2c46698e2c44b691e866aabd1e05e4"
format_stamp: "Formatted at 2026-08-12 11:13:49 on dist-test-slave-drl0"
server_key: "05083fb7e765a519822ebfc37384ab56"
server_key_iv: "1a96d350f388056207e9a802e6ea70b1"
server_key_version: "encryptionkey@0"
I20260812 11:13:49.399037 11989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:49.436420 11989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:49.439855 11989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:49.440061 11989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:49.440676 11989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:13:49.441565 11989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:13:49.441640 11989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:49.441712 11989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:13:49.441768 11989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:49.453006 11989 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:46533
I20260812 11:13:49.453069 12197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:46533 every 8 connection(s)
I20260812 11:13:49.454066 11989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:13:49.459935 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 11989
I20260812 11:13:49.460044 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:13:49.460328 17587 external_mini_cluster.cc:1428] Setting key 2f22159dcd4f8f33a80495e959ae817c
Aug 12 11:13:49 dist-test-slave-drl0 krb5kdc[11623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533229, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:13:49.470709 11742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:49.456139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:58649 (local address 127.17.44.254:46559)
0812 11:13:49.456275 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:49.456280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:49.457101 (+ 821us) server_negotiation.cc:407] Connection header received
0812 11:13:49.458389 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:49.458392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:49.458447 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:49.458560 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:49.459923 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.460748 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.462130 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.462548 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.462761 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:49.465541 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:49.465545 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:49.465548 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:13:49.465581 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:49.467724 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:49.468635 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:49.468638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:49.468639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:49.468804 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:49.469314 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:49.469318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:49.469320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:49.469527 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:49.469641 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:49.470451 (+ 810us) server_negotiation.cc:299] Negotiation successful
0812 11:13:49.470576 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 11:13:49.472069 12204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:49.456465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:46559 (local address 127.17.44.194:58649)
0812 11:13:49.456943 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:49.456992 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:49.458188 (+ 1196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:49.458720 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:49.458726 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:49.459156 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:13:49.459726 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:49.459748 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.460892 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.460895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:49.461901 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:49.461916 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.462796 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.462799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:49.463020 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:49.463659 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:13:49.463675 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:13:49.465367 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:13:49.467909 (+ 2542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:49.467920 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:49.467942 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:49.468433 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:49.468942 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:49.468949 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:49.468955 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:49.469164 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:49.469717 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:13:49.469727 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:13:49.470131 (+ 404us) client_negotiation.cc:769] Sending connection context
0812 11:13:49.470392 (+ 261us) client_negotiation.cc:240] Negotiation successful
0812 11:13:49.470917 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":117,"threads_started":1}
I20260812 11:13:49.473383 12198 heartbeater.cc:347] Connected to a master server at 127.17.44.254:46559
WARNING: no policy specified for kudu/127.17.44.195@KRBTEST.COM; defaulting to no policy
I20260812 11:13:49.473658 12198 heartbeater.cc:464] Registering TS with master...
I20260812 11:13:49.474306 12198 heartbeater.cc:511] Master 127.17.44.254:46559 requested a full tablet report, sending...
I20260812 11:13:49.475556 11686 ts_manager.cc:194] Registered new tserver with Master: 3a2c46698e2c44b691e866aabd1e05e4 (127.17.44.194:46533)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.195@KRBTEST.COM" created.
I20260812 11:13:49.476213 11686 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.194@KRBTEST.COM'} at 127.17.44.194:58649
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
I20260812 11:13:49.495428 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:46559
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:44683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:13:49.617740 12209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:49.617985 12209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:49.618027 12209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:49.622208 12209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:49.622336 12209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:49.622418 12209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:13:49.628100 12209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:44683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:46559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:49.629112 12209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:49.630373 12209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:49.640404 12220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:49.640415 12222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:49.640419 12219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:49.640663 12209 server_base.cc:1034] running on GCE node
I20260812 11:13:49.641130 12209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:49.641700 12209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:49.642889 12209 hybrid_clock.cc:648] HybridClock initialized: now 1786533229642868 us; error 32 us; skew 500 ppm
Aug 12 11:13:49 dist-test-slave-drl0 krb5kdc[11623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533229, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:13:49.646250 12209 init.cc:377] Logged in from keytab as kudu/127.17.44.195@KRBTEST.COM (short username kudu)
I20260812 11:13:49.647646 12209 webserver.cc:460] Webserver started at http://127.17.44.195:46785/ using document root <none> and password file <none>
I20260812 11:13:49.648201 12209 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:49.648275 12209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:49.648454 12209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:13:49.650599 12209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "89872207f263480da1f08d41617bf251"
format_stamp: "Formatted at 2026-08-12 11:13:49 on dist-test-slave-drl0"
server_key: "b0209f06424f2ed378c694cbba4ee963"
server_key_iv: "1b4ba68a98b080acc523ea2c6ef130cc"
server_key_version: "encryptionkey@0"
I20260812 11:13:49.651332 12209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "89872207f263480da1f08d41617bf251"
format_stamp: "Formatted at 2026-08-12 11:13:49 on dist-test-slave-drl0"
server_key: "b0209f06424f2ed378c694cbba4ee963"
server_key_iv: "1b4ba68a98b080acc523ea2c6ef130cc"
server_key_version: "encryptionkey@0"
I20260812 11:13:49.655930 12209 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 11:13:49.658643 12233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:49.660326 12209 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 11:13:49.660429 12209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "89872207f263480da1f08d41617bf251"
format_stamp: "Formatted at 2026-08-12 11:13:49 on dist-test-slave-drl0"
server_key: "b0209f06424f2ed378c694cbba4ee963"
server_key_iv: "1b4ba68a98b080acc523ea2c6ef130cc"
server_key_version: "encryptionkey@0"
I20260812 11:13:49.660568 12209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:49.685957 12209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:49.690524 12209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:49.690807 12209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:49.691763 12209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:13:49.692821 12209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:13:49.692876 12209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:49.692917 12209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:13:49.692934 12209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:49.702440 12209 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:42179
I20260812 11:13:49.702500 12406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:42179 every 8 connection(s)
I20260812 11:13:49.703400 12209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:13:49.704023 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 12209
I20260812 11:13:49.704180 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:13:49.704493 17587 external_mini_cluster.cc:1428] Setting key 9a0ab52c686504f952ecbee19064c349
Aug 12 11:13:49 dist-test-slave-drl0 krb5kdc[11623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533229, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:13:49.719722 11742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:49.706191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:43807 (local address 127.17.44.254:46559)
0812 11:13:49.706339 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:49.706343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:49.707762 (+ 1419us) server_negotiation.cc:407] Connection header received
0812 11:13:49.709626 (+ 1864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:49.709629 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:49.709680 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:49.709760 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:49.711091 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.711647 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.712873 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.713104 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.713232 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:49.715813 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:49.715816 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:49.715819 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:13:49.715841 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:49.717259 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:49.717897 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:49.717900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:49.717902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:49.717984 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:49.718358 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:49.718362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:49.718363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:49.718591 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:49.718701 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:49.719446 (+ 745us) server_negotiation.cc:299] Negotiation successful
0812 11:13:49.719567 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 11:13:49.720683 12410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:49.706811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:46559 (local address 127.17.44.195:43807)
0812 11:13:49.707563 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:49.707633 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:49.709299 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:49.709926 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:49.709933 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:49.710349 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:13:49.710881 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:49.710896 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.711803 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.711806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:49.712728 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:49.712738 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.713283 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.713288 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:49.713474 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:49.714089 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:13:49.714107 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:13:49.715631 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:13:49.717416 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:49.717422 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:49.717439 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:49.717763 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:49.718090 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:49.718092 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:49.718094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:49.718242 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:49.718780 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:13:49.718788 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:13:49.719121 (+ 333us) client_negotiation.cc:769] Sending connection context
0812 11:13:49.719391 (+ 270us) client_negotiation.cc:240] Negotiation successful
0812 11:13:49.719685 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":180,"threads_started":1}
I20260812 11:13:49.722072 12408 heartbeater.cc:347] Connected to a master server at 127.17.44.254:46559
I20260812 11:13:49.722301 12408 heartbeater.cc:464] Registering TS with master...
I20260812 11:13:49.722777 12408 heartbeater.cc:511] Master 127.17.44.254:46559 requested a full tablet report, sending...
I20260812 11:13:49.724133 11686 ts_manager.cc:194] Registered new tserver with Master: 89872207f263480da1f08d41617bf251 (127.17.44.195:42179)
I20260812 11:13:49.724937 11686 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.195@KRBTEST.COM'} at 127.17.44.195:43807
I20260812 11:13:49.732847 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:13:49.744585 11742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:49.735809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34896 (local address 127.17.44.254:46559)
0812 11:13:49.735972 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:49.735975 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:49.736194 (+ 219us) server_negotiation.cc:407] Connection header received
0812 11:13:49.736271 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:49.736273 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:49.736316 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:49.736396 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:49.737124 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.737645 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.738532 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.738742 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.738844 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:49.740526 (+ 1682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:49.740529 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:49.740531 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:13:49.740552 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:49.742408 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:49.743171 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:49.743181 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:49.743183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:49.743267 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:49.743645 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:49.743649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:49.743650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:49.743951 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:49.744153 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:49.744341 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 11:13:49.744465 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 11:13:49.748505 11686 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34896:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:13:49.752566 11686 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:13:49.768316 12423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:49.761612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60114 (local address 127.17.44.193:41619)
0812 11:13:49.761963 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:49.761970 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:49.762098 (+ 128us) server_negotiation.cc:407] Connection header received
0812 11:13:49.762322 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:49.762330 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:49.762553 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:49.762707 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:49.763243 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.764537 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.766619 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.767561 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.767711 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:49.767893 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:49.768024 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 11:13:49.768135 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":73,"threads_started":1}
I20260812 11:13:49.768306 12424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:49.761538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:46533 (local address 127.0.0.1:48912)
0812 11:13:49.762741 (+ 1203us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:49.762755 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:49.762838 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:49.763554 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:49.763557 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:49.763576 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:49.763688 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:49.763698 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.765000 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.765004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:49.766479 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:49.766490 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.767871 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.767875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:49.767997 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:49.768025 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 11:13:49.768085 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 11:13:49.768151 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1143,"mutex_wait_us":75,"thread_start_us":107,"threads_started":1}
I20260812 11:13:49.768316 12421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:49.761508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:41619 (local address 127.0.0.1:60114)
0812 11:13:49.761903 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:49.761948 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:49.762115 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:49.762901 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:49.762910 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:49.762962 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:49.763102 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:49.763110 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.764657 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.764661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:49.766479 (+ 1818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:49.766490 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.767697 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.767701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:49.767812 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:49.767857 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 11:13:49.767995 (+ 138us) client_negotiation.cc:240] Negotiation successful
0812 11:13:49.768130 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":127,"thread_start_us":70,"threads_started":1}
I20260812 11:13:49.768905 12422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:49.762306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:42179 (local address 127.0.0.1:58570)
0812 11:13:49.762895 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:49.762909 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:49.763075 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:49.763929 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:49.763932 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:49.763951 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:49.764067 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:49.764074 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.765496 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.765500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:49.766655 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:49.766666 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.768316 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.768322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:49.768423 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:49.768454 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 11:13:49.768530 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 11:13:49.768762 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"mutex_wait_us":1,"thread_start_us":117,"threads_started":1}
I20260812 11:13:49.769153 12429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:49.762336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48912 (local address 127.17.44.194:46533)
0812 11:13:49.763116 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:49.763128 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:49.763149 (+ 21us) server_negotiation.cc:407] Connection header received
0812 11:13:49.763225 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:49.763230 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:49.763414 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:49.763576 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:49.763820 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.764845 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.766846 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.767718 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.768687 (+ 969us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:49.768834 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:49.768885 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 11:13:49.768985 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":143,"threads_started":1}
I20260812 11:13:49.769320 12430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:49.762537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58570 (local address 127.17.44.195:42179)
0812 11:13:49.763368 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:49.763381 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:49.763402 (+ 21us) server_negotiation.cc:407] Connection header received
0812 11:13:49.763471 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:49.763475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:49.763642 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:49.763962 (+ 320us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:49.764203 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.765324 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.766893 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.768026 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.768122 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:49.768250 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:49.769068 (+ 818us) server_negotiation.cc:299] Negotiation successful
0812 11:13:49.769167 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":108,"threads_started":1}
I20260812 11:13:49.771867 11875 tablet_service.cc:1467] Processing CreateTablet for tablet aa351d9a5d5a49439df965d25d04ca9a (DEFAULT_TABLE table=test-table [id=d296c537215c43d1b79e8bd8ced4663d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:13:49.771932 12095 tablet_service.cc:1467] Processing CreateTablet for tablet aa351d9a5d5a49439df965d25d04ca9a (DEFAULT_TABLE table=test-table [id=d296c537215c43d1b79e8bd8ced4663d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:13:49.771931 12310 tablet_service.cc:1467] Processing CreateTablet for tablet aa351d9a5d5a49439df965d25d04ca9a (DEFAULT_TABLE table=test-table [id=d296c537215c43d1b79e8bd8ced4663d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:13:49.773005 12095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa351d9a5d5a49439df965d25d04ca9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:49.773005 12310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa351d9a5d5a49439df965d25d04ca9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:49.773005 11875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa351d9a5d5a49439df965d25d04ca9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:49.779588 12431 tablet_bootstrap.cc:492] T aa351d9a5d5a49439df965d25d04ca9a P d864eb97813c41e3ac010aac6522335c: Bootstrap starting.
I20260812 11:13:49.780400 12432 tablet_bootstrap.cc:492] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4: Bootstrap starting.
I20260812 11:13:49.781605 12433 tablet_bootstrap.cc:492] T aa351d9a5d5a49439df965d25d04ca9a P 89872207f263480da1f08d41617bf251: Bootstrap starting.
I20260812 11:13:49.781862 12431 tablet_bootstrap.cc:654] T aa351d9a5d5a49439df965d25d04ca9a P d864eb97813c41e3ac010aac6522335c: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:49.782555 12432 tablet_bootstrap.cc:654] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:49.782750 12431 log.cc:824] T aa351d9a5d5a49439df965d25d04ca9a P d864eb97813c41e3ac010aac6522335c: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:49.783839 12433 tablet_bootstrap.cc:654] T aa351d9a5d5a49439df965d25d04ca9a P 89872207f263480da1f08d41617bf251: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:49.783912 12432 log.cc:824] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:49.785104 12433 log.cc:824] T aa351d9a5d5a49439df965d25d04ca9a P 89872207f263480da1f08d41617bf251: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:49.785876 12432 tablet_bootstrap.cc:492] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4: No bootstrap required, opened a new log
I20260812 11:13:49.785863 12431 tablet_bootstrap.cc:492] T aa351d9a5d5a49439df965d25d04ca9a P d864eb97813c41e3ac010aac6522335c: No bootstrap required, opened a new log
I20260812 11:13:49.786100 12432 ts_tablet_manager.cc:1397] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 11:13:49.786108 12431 ts_tablet_manager.cc:1397] T aa351d9a5d5a49439df965d25d04ca9a P d864eb97813c41e3ac010aac6522335c: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 11:13:49.787128 12433 tablet_bootstrap.cc:492] T aa351d9a5d5a49439df965d25d04ca9a P 89872207f263480da1f08d41617bf251: No bootstrap required, opened a new log
I20260812 11:13:49.787315 12433 ts_tablet_manager.cc:1397] T aa351d9a5d5a49439df965d25d04ca9a P 89872207f263480da1f08d41617bf251: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:13:49.790174 12432 raft_consensus.cc:348] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864eb97813c41e3ac010aac6522335c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 41619 } } peers { permanent_uuid: "3a2c46698e2c44b691e866aabd1e05e4" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46533 } } peers { permanent_uuid: "89872207f263480da1f08d41617bf251" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 42179 } }
I20260812 11:13:49.790669 12432 raft_consensus.cc:374] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:49.790769 12432 raft_consensus.cc:729] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a2c46698e2c44b691e866aabd1e05e4, State: Initialized, Role: FOLLOWER
I20260812 11:13:49.790887 12431 raft_consensus.cc:348] T aa351d9a5d5a49439df965d25d04ca9a P d864eb97813c41e3ac010aac6522335c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864eb97813c41e3ac010aac6522335c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 41619 } } peers { permanent_uuid: "3a2c46698e2c44b691e866aabd1e05e4" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46533 } } peers { permanent_uuid: "89872207f263480da1f08d41617bf251" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 42179 } }
I20260812 11:13:49.791249 12431 raft_consensus.cc:374] T aa351d9a5d5a49439df965d25d04ca9a P d864eb97813c41e3ac010aac6522335c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:49.791350 12431 raft_consensus.cc:729] T aa351d9a5d5a49439df965d25d04ca9a P d864eb97813c41e3ac010aac6522335c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d864eb97813c41e3ac010aac6522335c, State: Initialized, Role: FOLLOWER
I20260812 11:13:49.791235 12433 raft_consensus.cc:348] T aa351d9a5d5a49439df965d25d04ca9a P 89872207f263480da1f08d41617bf251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864eb97813c41e3ac010aac6522335c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 41619 } } peers { permanent_uuid: "3a2c46698e2c44b691e866aabd1e05e4" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46533 } } peers { permanent_uuid: "89872207f263480da1f08d41617bf251" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 42179 } }
I20260812 11:13:49.791493 12433 raft_consensus.cc:374] T aa351d9a5d5a49439df965d25d04ca9a P 89872207f263480da1f08d41617bf251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:49.791435 12432 consensus_queue.cc:260] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864eb97813c41e3ac010aac6522335c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 41619 } } peers { permanent_uuid: "3a2c46698e2c44b691e866aabd1e05e4" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46533 } } peers { permanent_uuid: "89872207f263480da1f08d41617bf251" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 42179 } }
I20260812 11:13:49.791563 12433 raft_consensus.cc:729] T aa351d9a5d5a49439df965d25d04ca9a P 89872207f263480da1f08d41617bf251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89872207f263480da1f08d41617bf251, State: Initialized, Role: FOLLOWER
I20260812 11:13:49.791908 12431 consensus_queue.cc:260] T aa351d9a5d5a49439df965d25d04ca9a P d864eb97813c41e3ac010aac6522335c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864eb97813c41e3ac010aac6522335c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 41619 } } peers { permanent_uuid: "3a2c46698e2c44b691e866aabd1e05e4" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46533 } } peers { permanent_uuid: "89872207f263480da1f08d41617bf251" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 42179 } }
I20260812 11:13:49.791949 12433 consensus_queue.cc:260] T aa351d9a5d5a49439df965d25d04ca9a P 89872207f263480da1f08d41617bf251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864eb97813c41e3ac010aac6522335c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 41619 } } peers { permanent_uuid: "3a2c46698e2c44b691e866aabd1e05e4" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46533 } } peers { permanent_uuid: "89872207f263480da1f08d41617bf251" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 42179 } }
I20260812 11:13:49.792526 12198 heartbeater.cc:503] Master 127.17.44.254:46559 was elected leader, sending a full tablet report...
I20260812 11:13:49.792848 11982 heartbeater.cc:503] Master 127.17.44.254:46559 was elected leader, sending a full tablet report...
I20260812 11:13:49.792999 12433 ts_tablet_manager.cc:1428] T aa351d9a5d5a49439df965d25d04ca9a P 89872207f263480da1f08d41617bf251: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:13:49.793056 12431 ts_tablet_manager.cc:1428] T aa351d9a5d5a49439df965d25d04ca9a P d864eb97813c41e3ac010aac6522335c: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 11:13:49.793489 12408 heartbeater.cc:503] Master 127.17.44.254:46559 was elected leader, sending a full tablet report...
I20260812 11:13:49.794155 12432 ts_tablet_manager.cc:1428] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260812 11:13:49.888262 12437 raft_consensus.cc:482] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:13:49.888542 12437 raft_consensus.cc:504] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864eb97813c41e3ac010aac6522335c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 41619 } } peers { permanent_uuid: "3a2c46698e2c44b691e866aabd1e05e4" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46533 } } peers { permanent_uuid: "89872207f263480da1f08d41617bf251" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 42179 } }
I20260812 11:13:49.890080 12437 leader_election.cc:288] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d864eb97813c41e3ac010aac6522335c (127.17.44.193:41619), 89872207f263480da1f08d41617bf251 (127.17.44.195:42179)
I20260812 11:13:49.894727 12423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:49.890479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:49409 (local address 127.17.44.193:41619)
0812 11:13:49.890607 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:49.890612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:49.890628 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:13:49.890751 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:49.890754 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:49.890803 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:49.890886 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:49.891315 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.891976 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.893513 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.894129 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.894254 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:49.894317 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:49.894547 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 11:13:49.894625 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 11:13:49.894721 12204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:49.890325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:41619 (local address 127.17.44.194:49409)
0812 11:13:49.890520 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:49.890537 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:49.890631 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:49.891035 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:49.891039 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:49.891057 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:49.891208 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:49.891216 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.892092 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.892096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:49.893362 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:49.893373 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.894267 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.894270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:49.894395 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:49.894423 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 11:13:49.894512 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 11:13:49.894591 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":45}
I20260812 11:13:49.894858 12430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:49.890523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:58457 (local address 127.17.44.195:42179)
0812 11:13:49.890647 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:49.890652 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:49.891179 (+ 527us) server_negotiation.cc:407] Connection header received
0812 11:13:49.891257 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:49.891259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:49.891317 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:49.891390 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:49.891698 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.892380 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.893826 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.894257 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.894373 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:49.894424 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:49.894651 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 11:13:49.894737 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 11:13:49.895210 12442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:49.890331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:42179 (local address 127.17.44.194:58457)
0812 11:13:49.891063 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:49.891076 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:49.891172 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:49.891420 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:49.891430 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:49.891448 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:49.891582 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:49.891588 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.892511 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.892514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:49.893701 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:49.893709 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.894397 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.894401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:49.894506 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:49.894528 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 11:13:49.895005 (+ 477us) client_negotiation.cc:240] Negotiation successful
0812 11:13:49.895077 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"mutex_wait_us":339,"thread_start_us":154,"threads_started":1}
I20260812 11:13:49.895782 11906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa351d9a5d5a49439df965d25d04ca9a" candidate_uuid: "3a2c46698e2c44b691e866aabd1e05e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d864eb97813c41e3ac010aac6522335c" is_pre_election: true
I20260812 11:13:49.895977 12338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa351d9a5d5a49439df965d25d04ca9a" candidate_uuid: "3a2c46698e2c44b691e866aabd1e05e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89872207f263480da1f08d41617bf251" is_pre_election: true
I20260812 11:13:49.896106 11906 raft_consensus.cc:2454] T aa351d9a5d5a49439df965d25d04ca9a P d864eb97813c41e3ac010aac6522335c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a2c46698e2c44b691e866aabd1e05e4 in term 0.
I20260812 11:13:49.896245 12338 raft_consensus.cc:2454] T aa351d9a5d5a49439df965d25d04ca9a P 89872207f263480da1f08d41617bf251 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a2c46698e2c44b691e866aabd1e05e4 in term 0.
I20260812 11:13:49.896642 12020 leader_election.cc:302] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3a2c46698e2c44b691e866aabd1e05e4, 89872207f263480da1f08d41617bf251, d864eb97813c41e3ac010aac6522335c; no voters:
I20260812 11:13:49.896907 12437 raft_consensus.cc:2781] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:13:49.896989 12437 raft_consensus.cc:482] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:13:49.897027 12437 raft_consensus.cc:3037] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:49.898301 12437 raft_consensus.cc:504] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864eb97813c41e3ac010aac6522335c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 41619 } } peers { permanent_uuid: "3a2c46698e2c44b691e866aabd1e05e4" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46533 } } peers { permanent_uuid: "89872207f263480da1f08d41617bf251" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 42179 } }
I20260812 11:13:49.898644 12437 leader_election.cc:288] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4 [CANDIDATE]: Term 1 election: Requested vote from peers d864eb97813c41e3ac010aac6522335c (127.17.44.193:41619), 89872207f263480da1f08d41617bf251 (127.17.44.195:42179)
I20260812 11:13:49.899052 12338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa351d9a5d5a49439df965d25d04ca9a" candidate_uuid: "3a2c46698e2c44b691e866aabd1e05e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89872207f263480da1f08d41617bf251"
I20260812 11:13:49.899159 12338 raft_consensus.cc:3037] T aa351d9a5d5a49439df965d25d04ca9a P 89872207f263480da1f08d41617bf251 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:49.899291 11906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa351d9a5d5a49439df965d25d04ca9a" candidate_uuid: "3a2c46698e2c44b691e866aabd1e05e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d864eb97813c41e3ac010aac6522335c"
I20260812 11:13:49.899391 11906 raft_consensus.cc:3037] T aa351d9a5d5a49439df965d25d04ca9a P d864eb97813c41e3ac010aac6522335c [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:49.900344 12338 raft_consensus.cc:2454] T aa351d9a5d5a49439df965d25d04ca9a P 89872207f263480da1f08d41617bf251 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a2c46698e2c44b691e866aabd1e05e4 in term 1.
I20260812 11:13:49.900372 11906 raft_consensus.cc:2454] T aa351d9a5d5a49439df965d25d04ca9a P d864eb97813c41e3ac010aac6522335c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a2c46698e2c44b691e866aabd1e05e4 in term 1.
I20260812 11:13:49.900676 12021 leader_election.cc:302] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a2c46698e2c44b691e866aabd1e05e4, 89872207f263480da1f08d41617bf251; no voters:
I20260812 11:13:49.900880 12437 raft_consensus.cc:2781] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:13:49.901063 12437 raft_consensus.cc:686] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4 [term 1 LEADER]: Becoming Leader. State: Replica: 3a2c46698e2c44b691e866aabd1e05e4, State: Running, Role: LEADER
I20260812 11:13:49.901394 12437 consensus_queue.cc:237] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864eb97813c41e3ac010aac6522335c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 41619 } } peers { permanent_uuid: "3a2c46698e2c44b691e866aabd1e05e4" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46533 } } peers { permanent_uuid: "89872207f263480da1f08d41617bf251" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 42179 } }
I20260812 11:13:49.905011 11682 catalog_manager.cc:5495] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a2c46698e2c44b691e866aabd1e05e4 (127.17.44.194). New cstate: current_term: 1 leader_uuid: "3a2c46698e2c44b691e866aabd1e05e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d864eb97813c41e3ac010aac6522335c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 41619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a2c46698e2c44b691e866aabd1e05e4" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89872207f263480da1f08d41617bf251" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 42179 } health_report { overall_health: UNKNOWN } } }
I20260812 11:13:49.922770 12429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:49.918725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48924 (local address 127.17.44.194:46533)
0812 11:13:49.919027 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:49.919031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:49.919044 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:13:49.919112 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:49.919115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:49.919179 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:49.919282 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:13:49.919656 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.920231 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.921252 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:49.921485 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:49.921603 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:49.921921 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:13:49.922348 (+ 427us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:13:49.922483 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:49.922545 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 11:13:49.922603 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213}
I20260812 11:13:49.929885 11906 raft_consensus.cc:1261] T aa351d9a5d5a49439df965d25d04ca9a P d864eb97813c41e3ac010aac6522335c [term 1 FOLLOWER]: Refusing update from remote peer 3a2c46698e2c44b691e866aabd1e05e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:13:49.929885 12338 raft_consensus.cc:1261] T aa351d9a5d5a49439df965d25d04ca9a P 89872207f263480da1f08d41617bf251 [term 1 FOLLOWER]: Refusing update from remote peer 3a2c46698e2c44b691e866aabd1e05e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:13:49.930998 12443 consensus_queue.cc:1035] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89872207f263480da1f08d41617bf251" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 42179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:13:49.931393 12437 consensus_queue.cc:1035] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d864eb97813c41e3ac010aac6522335c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:13:49.941305 12450 mvcc.cc:205] Tried to move back new op lower bound from 7317640109780385792 to 7317640109678800896. Current Snapshot: MvccSnapshot[applied={T|T < 7317640109780385792}]
I20260812 11:13:49.941919 12448 mvcc.cc:205] Tried to move back new op lower bound from 7317640109780385792 to 7317640109678800896. Current Snapshot: MvccSnapshot[applied={T|T < 7317640109780385792}]
I20260812 11:13:49.952627 11684 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:13:49.952823 11684 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:13:49.955600 11684 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 38d7bcdf74ee4d25890fb35ae561e6f1: Sending DeleteTablet for 3 replicas of tablet aa351d9a5d5a49439df965d25d04ca9a
I20260812 11:13:49.956429 12095 tablet_service.cc:1514] Processing DeleteTablet for tablet aa351d9a5d5a49439df965d25d04ca9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:13:49 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:48912
I20260812 11:13:49.956425 11875 tablet_service.cc:1514] Processing DeleteTablet for tablet aa351d9a5d5a49439df965d25d04ca9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:13:49 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:60114
I20260812 11:13:49.956770 12310 tablet_service.cc:1514] Processing DeleteTablet for tablet aa351d9a5d5a49439df965d25d04ca9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:13:49 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:58570
I20260812 11:13:49.956951 12465 tablet_replica.cc:330] T aa351d9a5d5a49439df965d25d04ca9a P d864eb97813c41e3ac010aac6522335c: stopping tablet replica
I20260812 11:13:49.957240 12469 tablet_replica.cc:330] T aa351d9a5d5a49439df965d25d04ca9a P 89872207f263480da1f08d41617bf251: stopping tablet replica
I20260812 11:13:49.957274 12465 raft_consensus.cc:2229] T aa351d9a5d5a49439df965d25d04ca9a P d864eb97813c41e3ac010aac6522335c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:13:49.957702 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 11778
I20260812 11:13:49.957756 12469 raft_consensus.cc:2229] T aa351d9a5d5a49439df965d25d04ca9a P 89872207f263480da1f08d41617bf251 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:13:49.963294 12469 raft_consensus.cc:2258] T aa351d9a5d5a49439df965d25d04ca9a P 89872207f263480da1f08d41617bf251 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:13:49.975973 12469 ts_tablet_manager.cc:1905] T aa351d9a5d5a49439df965d25d04ca9a P 89872207f263480da1f08d41617bf251: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:13:49.980348 12469 ts_tablet_manager.cc:1918] T aa351d9a5d5a49439df965d25d04ca9a P 89872207f263480da1f08d41617bf251: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:13:49.980579 12469 log.cc:1196] T aa351d9a5d5a49439df965d25d04ca9a P 89872207f263480da1f08d41617bf251: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/aa351d9a5d5a49439df965d25d04ca9a
I20260812 11:13:49.981175 12469 ts_tablet_manager.cc:1939] T aa351d9a5d5a49439df965d25d04ca9a P 89872207f263480da1f08d41617bf251: Deleting consensus metadata
I20260812 11:13:49.957710 12465 raft_consensus.cc:2258] T aa351d9a5d5a49439df965d25d04ca9a P d864eb97813c41e3ac010aac6522335c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:13:49.984218 11660 catalog_manager.cc:4841] TS 89872207f263480da1f08d41617bf251 (127.17.44.195:42179): tablet aa351d9a5d5a49439df965d25d04ca9a (table test-table [id=d296c537215c43d1b79e8bd8ced4663d]) successfully deleted
I20260812 11:13:49.984444 12464 tablet_replica.cc:330] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4: stopping tablet replica
I20260812 11:13:49.984815 12464 raft_consensus.cc:2229] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:13:49.985287 12464 raft_consensus.cc:2258] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:13:49.986651 12464 ts_tablet_manager.cc:1905] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:13:49.989365 12464 ts_tablet_manager.cc:1918] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:13:49.989465 12464 log.cc:1196] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/aa351d9a5d5a49439df965d25d04ca9a
I20260812 11:13:49.989780 12464 ts_tablet_manager.cc:1939] T aa351d9a5d5a49439df965d25d04ca9a P 3a2c46698e2c44b691e866aabd1e05e4: Deleting consensus metadata
I20260812 11:13:49.991011 11664 catalog_manager.cc:4841] TS 3a2c46698e2c44b691e866aabd1e05e4 (127.17.44.194:46533): tablet aa351d9a5d5a49439df965d25d04ca9a (table test-table [id=d296c537215c43d1b79e8bd8ced4663d]) successfully deleted
W20260812 11:13:49.991936 11658 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.44.193:41619 (error 108)
I20260812 11:13:49.992096 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 11989
I20260812 11:13:49.994349 12422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:49.992855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:41619 (local address 127.0.0.1:60124)
0812 11:13:49.994170 (+ 1315us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:49.994251 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.193:41619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1221}
W20260812 11:13:49.994570 11658 catalog_manager.cc:4584] TS d864eb97813c41e3ac010aac6522335c (127.17.44.193:41619): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet aa351d9a5d5a49439df965d25d04ca9a: Network error: Client connection negotiation failed: client connection to 127.17.44.193:41619: connect: Connection refused (error 111)
I20260812 11:13:50.004492 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 12209
I20260812 11:13:50.021237 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 11638
2026-08-12T11:13:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2769 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T11:13:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:13:50Z Disabled control of system clock
I20260812 11:13:50.058315 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:33129
--webserver_interface=127.17.44.254
--webserver_port=0
--builtin_ntp_servers=127.17.44.212:43503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:33129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 11:13:50.174798 12476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:50.175060 12476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:50.175096 12476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:50.179029 12476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:13:50.179075 12476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:50.179090 12476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:50.179107 12476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:13:50.179122 12476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:13:50.183990 12476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:43503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:33129
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:33129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:50.184937 12476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:50.186247 12476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:50.194886 12488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:50.195138 12476 server_base.cc:1034] running on GCE node
W20260812 11:13:50.194895 12484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:50.194895 12486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:50.195804 12476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:50.197011 12476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:50.198203 12476 hybrid_clock.cc:648] HybridClock initialized: now 1786533230198168 us; error 46 us; skew 500 ppm
I20260812 11:13:50.200201 12476 webserver.cc:460] Webserver started at http://127.17.44.254:35109/ using document root <none> and password file <none>
I20260812 11:13:50.200937 12476 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:50.201020 12476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:50.201252 12476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:13:50.203547 12476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "0975ff41a46349ec8809cabb2fb5dab7"
format_stamp: "Formatted at 2026-08-12 11:13:50 on dist-test-slave-drl0"
server_key: "8d8ae60afa92fb2a88aed4715113b3a0"
server_key_iv: "273ef25d0c0c1db2fea4ced3bf6287fe"
server_key_version: "encryptionkey@0"
I20260812 11:13:50.204198 12476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "0975ff41a46349ec8809cabb2fb5dab7"
format_stamp: "Formatted at 2026-08-12 11:13:50 on dist-test-slave-drl0"
server_key: "8d8ae60afa92fb2a88aed4715113b3a0"
server_key_iv: "273ef25d0c0c1db2fea4ced3bf6287fe"
server_key_version: "encryptionkey@0"
I20260812 11:13:50.208135 12476 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260812 11:13:50.210995 12498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:50.212343 12476 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 11:13:50.212520 12476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "0975ff41a46349ec8809cabb2fb5dab7"
format_stamp: "Formatted at 2026-08-12 11:13:50 on dist-test-slave-drl0"
server_key: "8d8ae60afa92fb2a88aed4715113b3a0"
server_key_iv: "273ef25d0c0c1db2fea4ced3bf6287fe"
server_key_version: "encryptionkey@0"
I20260812 11:13:50.212638 12476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:50.231592 12476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:50.232317 12476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:50.232532 12476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:50.240444 12476 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:33129
I20260812 11:13:50.240571 12583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:33129 every 8 connection(s)
I20260812 11:13:50.241415 12476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:13:50.244582 12585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:50.244684 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 12476
I20260812 11:13:50.244841 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:13:50.245132 17587 external_mini_cluster.cc:1428] Setting key a7a0cc20d0b8d100a284fe5b7b39998a
I20260812 11:13:50.252491 12585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0975ff41a46349ec8809cabb2fb5dab7: Bootstrap starting.
I20260812 11:13:50.254851 12585 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0975ff41a46349ec8809cabb2fb5dab7: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:50.255505 12589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:50.246557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57008 (local address 127.17.44.254:33129)
0812 11:13:50.247676 (+ 1119us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:50.247695 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:50.247738 (+ 43us) server_negotiation.cc:407] Connection header received
0812 11:13:50.248900 (+ 1162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:50.248919 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:50.249294 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:50.249641 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:50.250069 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.251280 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.252184 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.252685 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.252850 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:50.253810 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:50.253814 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:50.253826 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:13:50.254192 (+ 366us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:50.254244 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:13:50.254263 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:50.254381 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:50.254505 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 11:13:50.254752 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":127,"threads_started":1}
I20260812 11:13:50.255693 12585 log.cc:824] T 00000000000000000000000000000000 P 0975ff41a46349ec8809cabb2fb5dab7: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:50.257550 12585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0975ff41a46349ec8809cabb2fb5dab7: No bootstrap required, opened a new log
I20260812 11:13:50.261021 12585 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0975ff41a46349ec8809cabb2fb5dab7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0975ff41a46349ec8809cabb2fb5dab7" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33129 } }
I20260812 11:13:50.261278 12585 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0975ff41a46349ec8809cabb2fb5dab7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:50.261366 12585 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0975ff41a46349ec8809cabb2fb5dab7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0975ff41a46349ec8809cabb2fb5dab7, State: Initialized, Role: FOLLOWER
I20260812 11:13:50.262019 12585 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0975ff41a46349ec8809cabb2fb5dab7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0975ff41a46349ec8809cabb2fb5dab7" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33129 } }
I20260812 11:13:50.262358 12585 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0975ff41a46349ec8809cabb2fb5dab7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:13:50.262477 12585 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0975ff41a46349ec8809cabb2fb5dab7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:13:50.262660 12585 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0975ff41a46349ec8809cabb2fb5dab7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:50.264277 12585 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0975ff41a46349ec8809cabb2fb5dab7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0975ff41a46349ec8809cabb2fb5dab7" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33129 } }
I20260812 11:13:50.264743 12585 leader_election.cc:302] T 00000000000000000000000000000000 P 0975ff41a46349ec8809cabb2fb5dab7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0975ff41a46349ec8809cabb2fb5dab7; no voters:
I20260812 11:13:50.265257 12585 leader_election.cc:288] T 00000000000000000000000000000000 P 0975ff41a46349ec8809cabb2fb5dab7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:13:50.265463 12591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0975ff41a46349ec8809cabb2fb5dab7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:13:50.265801 12591 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0975ff41a46349ec8809cabb2fb5dab7 [term 1 LEADER]: Becoming Leader. State: Replica: 0975ff41a46349ec8809cabb2fb5dab7, State: Running, Role: LEADER
I20260812 11:13:50.266279 12591 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0975ff41a46349ec8809cabb2fb5dab7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0975ff41a46349ec8809cabb2fb5dab7" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33129 } }
I20260812 11:13:50.266602 12585 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0975ff41a46349ec8809cabb2fb5dab7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:13:50.268006 12593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0975ff41a46349ec8809cabb2fb5dab7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0975ff41a46349ec8809cabb2fb5dab7. Latest consensus state: current_term: 1 leader_uuid: "0975ff41a46349ec8809cabb2fb5dab7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0975ff41a46349ec8809cabb2fb5dab7" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33129 } } }
I20260812 11:13:50.268146 12593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0975ff41a46349ec8809cabb2fb5dab7 [sys.catalog]: This master's current role is: LEADER
I20260812 11:13:50.268512 12592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0975ff41a46349ec8809cabb2fb5dab7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0975ff41a46349ec8809cabb2fb5dab7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0975ff41a46349ec8809cabb2fb5dab7" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33129 } } }
I20260812 11:13:50.268649 12592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0975ff41a46349ec8809cabb2fb5dab7 [sys.catalog]: This master's current role is: LEADER
I20260812 11:13:50.268620 12603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:13:50.272150 12603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:13:50.277570 12603 catalog_manager.cc:1347] Generated new cluster ID: be19dabe0100408d9d371b4d58dbb6d2
I20260812 11:13:50.277642 12603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:13:50.292151 12603 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:13:50.293078 12603 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:13:50.299788 12603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0975ff41a46349ec8809cabb2fb5dab7: Generated new TSK 0
I20260812 11:13:50.300467 12603 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:13:50.307999 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:33129
--builtin_ntp_servers=127.17.44.212:43503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 11:13:50.427857 12615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:50.428134 12615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:50.428241 12615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:50.432687 12615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:50.432744 12615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:50.432832 12615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:13:50.437747 12615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:43503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:33129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:50.438829 12615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:50.440214 12615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:50.450222 12623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:50.450351 12628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:50.450565 12624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:50.450979 12615 server_base.cc:1034] running on GCE node
I20260812 11:13:50.451407 12615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:50.452067 12615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:50.453245 12615 hybrid_clock.cc:648] HybridClock initialized: now 1786533230453212 us; error 46 us; skew 500 ppm
I20260812 11:13:50.455438 12615 webserver.cc:460] Webserver started at http://127.17.44.193:34491/ using document root <none> and password file <none>
I20260812 11:13:50.456028 12615 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:50.456089 12615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:50.456313 12615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:13:50.458379 12615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "2af6dd25904349b7ac506f69091cef5d"
format_stamp: "Formatted at 2026-08-12 11:13:50 on dist-test-slave-drl0"
server_key: "efdfbb49190830153e518059fe069522"
server_key_iv: "a6cbe0a88ec515f2e1a168e688434b4f"
server_key_version: "encryptionkey@0"
I20260812 11:13:50.459066 12615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "2af6dd25904349b7ac506f69091cef5d"
format_stamp: "Formatted at 2026-08-12 11:13:50 on dist-test-slave-drl0"
server_key: "efdfbb49190830153e518059fe069522"
server_key_iv: "a6cbe0a88ec515f2e1a168e688434b4f"
server_key_version: "encryptionkey@0"
I20260812 11:13:50.462657 12615 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 11:13:50.465679 12635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:50.467425 12615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 11:13:50.467566 12615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "2af6dd25904349b7ac506f69091cef5d"
format_stamp: "Formatted at 2026-08-12 11:13:50 on dist-test-slave-drl0"
server_key: "efdfbb49190830153e518059fe069522"
server_key_iv: "a6cbe0a88ec515f2e1a168e688434b4f"
server_key_version: "encryptionkey@0"
I20260812 11:13:50.467681 12615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:50.508983 12615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:50.509773 12615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:50.509999 12615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:50.510630 12615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:13:50.511767 12615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:13:50.511822 12615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:50.511864 12615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:13:50.511880 12615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:50.521855 12615 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:39787
I20260812 11:13:50.521867 12808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:39787 every 8 connection(s)
I20260812 11:13:50.522773 12615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:13:50.526669 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 12615
I20260812 11:13:50.526773 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:13:50.527209 17587 external_mini_cluster.cc:1428] Setting key c5f5916333221a3f147baa73d42cbf08
I20260812 11:13:50.529206 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:33129
--builtin_ntp_servers=127.17.44.212:43503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 11:13:50.534044 12589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:50.524636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:50025 (local address 127.17.44.254:33129)
0812 11:13:50.524788 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:50.524792 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:50.525845 (+ 1053us) server_negotiation.cc:407] Connection header received
0812 11:13:50.527500 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:50.527504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:50.527566 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:50.527659 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:50.528897 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.529737 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.531566 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.532045 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.532176 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:50.533364 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:50.533371 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:50.533376 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:13:50.533394 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:50.533421 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:13:50.533434 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:50.533518 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:50.533731 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 11:13:50.533821 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 11:13:50.534778 12813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:50.525027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33129 (local address 127.17.44.193:50025)
0812 11:13:50.525692 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:50.525732 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:50.527266 (+ 1534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:50.527864 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:50.527873 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:50.527994 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:13:50.528569 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:50.528586 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.529990 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.529996 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:50.531413 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:50.531423 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.532263 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.532266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:50.532434 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:50.533096 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:13:50.533098 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:13:50.533139 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:13:50.533202 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:13:50.533570 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:13:50.533579 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:13:50.533585 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 11:13:50.533701 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 11:13:50.533919 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":116,"threads_started":1}
I20260812 11:13:50.536168 12809 heartbeater.cc:347] Connected to a master server at 127.17.44.254:33129
I20260812 11:13:50.536407 12809 heartbeater.cc:464] Registering TS with master...
I20260812 11:13:50.536933 12809 heartbeater.cc:511] Master 127.17.44.254:33129 requested a full tablet report, sending...
I20260812 11:13:50.538463 12533 ts_manager.cc:194] Registered new tserver with Master: 2af6dd25904349b7ac506f69091cef5d (127.17.44.193:39787)
I20260812 11:13:50.539705 12533 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.44.193:50025
W20260812 11:13:50.655134 12814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:50.655428 12814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:50.655498 12814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:50.660316 12814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:50.660426 12814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:50.660523 12814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:13:50.668294 12814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:43503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:33129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:50.669526 12814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:50.671195 12814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:50.681409 12825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:50.681445 12829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:50.681485 12827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:50.681638 12814 server_base.cc:1034] running on GCE node
I20260812 11:13:50.682120 12814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:50.682734 12814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:50.683974 12814 hybrid_clock.cc:648] HybridClock initialized: now 1786533230683936 us; error 71 us; skew 500 ppm
I20260812 11:13:50.686144 12814 webserver.cc:460] Webserver started at http://127.17.44.194:38609/ using document root <none> and password file <none>
I20260812 11:13:50.686770 12814 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:50.687038 12814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:50.687466 12814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:13:50.689829 12814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "954e02e1e56b48c39059bc48d9f2396e"
format_stamp: "Formatted at 2026-08-12 11:13:50 on dist-test-slave-drl0"
server_key: "b84652bb0f79935dfdde32f28deb7a11"
server_key_iv: "1d0f57ae485c0b3d3634c9a9210febf3"
server_key_version: "encryptionkey@0"
I20260812 11:13:50.690423 12814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "954e02e1e56b48c39059bc48d9f2396e"
format_stamp: "Formatted at 2026-08-12 11:13:50 on dist-test-slave-drl0"
server_key: "b84652bb0f79935dfdde32f28deb7a11"
server_key_iv: "1d0f57ae485c0b3d3634c9a9210febf3"
server_key_version: "encryptionkey@0"
I20260812 11:13:50.694504 12814 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 11:13:50.697350 12838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:50.698597 12814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:13:50.698746 12814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "954e02e1e56b48c39059bc48d9f2396e"
format_stamp: "Formatted at 2026-08-12 11:13:50 on dist-test-slave-drl0"
server_key: "b84652bb0f79935dfdde32f28deb7a11"
server_key_iv: "1d0f57ae485c0b3d3634c9a9210febf3"
server_key_version: "encryptionkey@0"
I20260812 11:13:50.698875 12814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:50.726198 12814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:50.727026 12814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:50.727358 12814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:50.728042 12814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:13:50.729795 12814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:13:50.729924 12814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:50.730007 12814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:13:50.730063 12814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:50.740094 12814 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:44741
I20260812 11:13:50.740114 13019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:44741 every 8 connection(s)
I20260812 11:13:50.741030 12814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:13:50.748497 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 12814
I20260812 11:13:50.748677 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:13:50.748976 17587 external_mini_cluster.cc:1428] Setting key 926c78912553b977d7f418d8a7c1503b
I20260812 11:13:50.751740 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:33129
--builtin_ntp_servers=127.17.44.212:43503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 11:13:50.752086 12589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:50.743145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:40515 (local address 127.17.44.254:33129)
0812 11:13:50.743294 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:50.743298 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:50.744395 (+ 1097us) server_negotiation.cc:407] Connection header received
0812 11:13:50.745877 (+ 1482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:50.745880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:50.745952 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:50.746071 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:50.747024 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.747612 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.748954 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.749239 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.749353 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:50.751364 (+ 2011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:50.751376 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:50.751379 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:13:50.751394 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:50.751420 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:13:50.751428 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:50.751527 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:50.751732 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 11:13:50.751824 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 11:13:50.753046 13025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:50.743514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33129 (local address 127.17.44.194:40515)
0812 11:13:50.744223 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:50.744264 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:50.745651 (+ 1387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:50.746243 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:50.746250 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:50.746332 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:13:50.746837 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:50.746850 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.747743 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.747746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:50.748817 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:50.748829 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.749411 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.749415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:50.749636 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:50.750837 (+ 1201us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:13:50.750841 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:13:50.750932 (+ 91us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:13:50.751158 (+ 226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:13:50.751529 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:13:50.751543 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:13:50.751554 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 11:13:50.751718 (+ 164us) client_negotiation.cc:240] Negotiation successful
0812 11:13:50.752204 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":137,"threads_started":1}
I20260812 11:13:50.754551 13022 heartbeater.cc:347] Connected to a master server at 127.17.44.254:33129
I20260812 11:13:50.754877 13022 heartbeater.cc:464] Registering TS with master...
I20260812 11:13:50.755456 13022 heartbeater.cc:511] Master 127.17.44.254:33129 requested a full tablet report, sending...
I20260812 11:13:50.756714 12533 ts_manager.cc:194] Registered new tserver with Master: 954e02e1e56b48c39059bc48d9f2396e (127.17.44.194:44741)
I20260812 11:13:50.757301 12533 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.44.194:40515
W20260812 11:13:50.872126 13026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:50.872445 13026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:50.872484 13026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:50.876509 13026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:50.876561 13026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:50.876613 13026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:13:50.881551 13026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:43503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:33129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:50.882591 13026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:50.884023 13026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:50.893883 13038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:50.893909 13036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:50.893970 13026 server_base.cc:1034] running on GCE node
W20260812 11:13:50.893909 13033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:50.894654 13026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:50.895287 13026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:50.896481 13026 hybrid_clock.cc:648] HybridClock initialized: now 1786533230896449 us; error 43 us; skew 500 ppm
I20260812 11:13:50.898335 13026 webserver.cc:460] Webserver started at http://127.17.44.195:44415/ using document root <none> and password file <none>
I20260812 11:13:50.898919 13026 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:50.899068 13026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:50.899287 13026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:13:50.901386 13026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "5228ff484b824a0ebbc1041d70f9818d"
format_stamp: "Formatted at 2026-08-12 11:13:50 on dist-test-slave-drl0"
server_key: "b0dfa70a583539272fa4943b6f890f5e"
server_key_iv: "324c9f4e1f79814573dbd89e815d969f"
server_key_version: "encryptionkey@0"
I20260812 11:13:50.902097 13026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "5228ff484b824a0ebbc1041d70f9818d"
format_stamp: "Formatted at 2026-08-12 11:13:50 on dist-test-slave-drl0"
server_key: "b0dfa70a583539272fa4943b6f890f5e"
server_key_iv: "324c9f4e1f79814573dbd89e815d969f"
server_key_version: "encryptionkey@0"
I20260812 11:13:50.906442 13026 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 11:13:50.909004 13046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:50.910269 13026 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:13:50.910418 13026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "5228ff484b824a0ebbc1041d70f9818d"
format_stamp: "Formatted at 2026-08-12 11:13:50 on dist-test-slave-drl0"
server_key: "b0dfa70a583539272fa4943b6f890f5e"
server_key_iv: "324c9f4e1f79814573dbd89e815d969f"
server_key_version: "encryptionkey@0"
I20260812 11:13:50.910530 13026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:50.921494 13026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:50.922113 13026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:50.922310 13026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:50.922883 13026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:13:50.923856 13026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:13:50.923928 13026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:50.923997 13026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:13:50.924048 13026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:50.936384 13026 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:40373
I20260812 11:13:50.936486 13220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:40373 every 8 connection(s)
I20260812 11:13:50.937597 13026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:13:50.941509 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 13026
I20260812 11:13:50.941638 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:13:50.941918 17587 external_mini_cluster.cc:1428] Setting key 9af58d20721f130d058ebe1145a32574
I20260812 11:13:50.949788 12589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:50.939879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:37327 (local address 127.17.44.254:33129)
0812 11:13:50.940046 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:50.940051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:50.941245 (+ 1194us) server_negotiation.cc:407] Connection header received
0812 11:13:50.943493 (+ 2248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:50.943496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:50.943558 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:50.943662 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:50.945646 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.946175 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.947578 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.947803 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.947897 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:50.949126 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:50.949129 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:50.949131 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:13:50.949144 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:50.949165 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:13:50.949173 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:50.949244 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:50.949598 (+ 354us) server_negotiation.cc:299] Negotiation successful
0812 11:13:50.949656 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 11:13:50.950755 13225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:50.940334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33129 (local address 127.17.44.195:37327)
0812 11:13:50.941035 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:50.941076 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:50.943026 (+ 1950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:50.944201 (+ 1175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:50.944221 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:50.944306 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:13:50.945478 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:50.945499 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.946331 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.946334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:50.947446 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:50.947456 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.947979 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.947982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:50.948129 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:50.948840 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:13:50.948841 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:13:50.948889 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:13:50.948955 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:13:50.949335 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:13:50.949352 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:13:50.949361 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 11:13:50.949650 (+ 289us) client_negotiation.cc:240] Negotiation successful
0812 11:13:50.949972 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":115,"threads_started":1}
I20260812 11:13:50.952088 13222 heartbeater.cc:347] Connected to a master server at 127.17.44.254:33129
I20260812 11:13:50.952349 13222 heartbeater.cc:464] Registering TS with master...
I20260812 11:13:50.953016 13222 heartbeater.cc:511] Master 127.17.44.254:33129 requested a full tablet report, sending...
I20260812 11:13:50.954700 12533 ts_manager.cc:194] Registered new tserver with Master: 5228ff484b824a0ebbc1041d70f9818d (127.17.44.195:40373)
I20260812 11:13:50.955519 12533 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.44.195:37327
I20260812 11:13:50.958634 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:13:50.965755 12589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:50.960906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57016 (local address 127.17.44.254:33129)
0812 11:13:50.961048 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:50.961052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:50.961236 (+ 184us) server_negotiation.cc:407] Connection header received
0812 11:13:50.961305 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:50.961307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:50.961345 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:50.961414 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:50.961770 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.962557 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.963669 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.963892 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.963986 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:50.965268 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:50.965271 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:50.965273 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:13:50.965285 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:50.965305 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:13:50.965312 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:50.965392 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:50.965591 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 11:13:50.965642 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 11:13:50.969732 12533 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57016:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:13:50.973258 12533 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:13:50.991254 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:50.984153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:44741 (local address 127.0.0.1:48178)
0812 11:13:50.985050 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:50.985122 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:50.985291 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:50.986180 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:50.986183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:50.986226 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:50.986664 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:50.986673 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.988424 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.988430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:50.989381 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:50.989391 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.990505 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.990511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:50.990633 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:50.990675 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 11:13:50.990830 (+ 155us) client_negotiation.cc:240] Negotiation successful
0812 11:13:50.991002 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"mutex_wait_us":169,"spinlock_wait_cycles":7936,"thread_start_us":154,"threads_started":1}
I20260812 11:13:50.991487 13235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:50.984166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42276 (local address 127.17.44.193:39787)
0812 11:13:50.984680 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:50.984691 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:50.985236 (+ 545us) server_negotiation.cc:407] Connection header received
0812 11:13:50.985499 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:50.985507 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:50.985809 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:50.985937 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:50.986855 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.988093 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.989852 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.990461 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.990544 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:50.990768 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:50.991170 (+ 402us) server_negotiation.cc:299] Negotiation successful
0812 11:13:50.991304 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":95,"threads_started":1}
I20260812 11:13:50.991891 13239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:50.983848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:39787 (local address 127.0.0.1:42276)
0812 11:13:50.985050 (+ 1202us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:50.985122 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:50.985301 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:50.986175 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:50.986181 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:50.986224 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:50.986661 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:50.986673 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.988232 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.988235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:50.989684 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:50.989697 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.990867 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.990870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:50.990987 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:50.991008 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 11:13:50.991651 (+ 643us) client_negotiation.cc:240] Negotiation successful
0812 11:13:50.991746 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":963,"mutex_wait_us":379,"spinlock_wait_cycles":5760,"thread_start_us":94,"threads_started":1}
I20260812 11:13:50.992377 13243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:50.985030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42440 (local address 127.17.44.195:40373)
0812 11:13:50.985474 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:50.985486 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:50.985701 (+ 215us) server_negotiation.cc:407] Connection header received
0812 11:13:50.985801 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:50.985808 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:50.986017 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:50.986196 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:50.986655 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.987938 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.989862 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.990736 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.990973 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:50.991296 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:50.992040 (+ 744us) server_negotiation.cc:299] Negotiation successful
0812 11:13:50.992175 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":116,"threads_started":1}
I20260812 11:13:50.992367 13240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:50.984460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48178 (local address 127.17.44.194:44741)
0812 11:13:50.985700 (+ 1240us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:50.985713 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:50.985737 (+ 24us) server_negotiation.cc:407] Connection header received
0812 11:13:50.985803 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:50.985807 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:50.986017 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:50.986196 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:50.987115 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.988314 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.989530 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.990329 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.990456 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:50.990728 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:50.992064 (+ 1336us) server_negotiation.cc:299] Negotiation successful
0812 11:13:50.992194 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1127,"thread_start_us":124,"threads_started":1}
I20260812 11:13:50.992615 13241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:50.984605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:40373 (local address 127.0.0.1:42440)
0812 11:13:50.985564 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:50.985578 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:50.985675 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:50.986281 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:50.986284 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:50.986294 (+ 10us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:50.986512 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:50.986522 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.988087 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.988091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:50.989684 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:50.989697 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:50.991369 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:50.991373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:50.991479 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:50.991502 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 11:13:50.991724 (+ 222us) client_negotiation.cc:240] Negotiation successful
0812 11:13:50.991824 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":883,"mutex_wait_us":124,"thread_start_us":84,"threads_started":1}
I20260812 11:13:50.995393 12709 tablet_service.cc:1467] Processing CreateTablet for tablet a85a23f84dbf477c9287c800182af22f (DEFAULT_TABLE table=test-table [id=2cffd59b65a34ff8acfe185425fd7233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:13:50.995388 13097 tablet_service.cc:1467] Processing CreateTablet for tablet a85a23f84dbf477c9287c800182af22f (DEFAULT_TABLE table=test-table [id=2cffd59b65a34ff8acfe185425fd7233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:13:50.995388 12911 tablet_service.cc:1467] Processing CreateTablet for tablet a85a23f84dbf477c9287c800182af22f (DEFAULT_TABLE table=test-table [id=2cffd59b65a34ff8acfe185425fd7233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:13:50.996510 12911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a85a23f84dbf477c9287c800182af22f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:50.996520 12709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a85a23f84dbf477c9287c800182af22f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:50.996520 13097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a85a23f84dbf477c9287c800182af22f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:51.004632 13245 tablet_bootstrap.cc:492] T a85a23f84dbf477c9287c800182af22f P 5228ff484b824a0ebbc1041d70f9818d: Bootstrap starting.
I20260812 11:13:51.005246 13244 tablet_bootstrap.cc:492] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d: Bootstrap starting.
I20260812 11:13:51.006187 13249 tablet_bootstrap.cc:492] T a85a23f84dbf477c9287c800182af22f P 954e02e1e56b48c39059bc48d9f2396e: Bootstrap starting.
I20260812 11:13:51.008931 13245 tablet_bootstrap.cc:654] T a85a23f84dbf477c9287c800182af22f P 5228ff484b824a0ebbc1041d70f9818d: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:51.008934 13244 tablet_bootstrap.cc:654] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:51.009096 13249 tablet_bootstrap.cc:654] T a85a23f84dbf477c9287c800182af22f P 954e02e1e56b48c39059bc48d9f2396e: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:51.009917 13249 log.cc:824] T a85a23f84dbf477c9287c800182af22f P 954e02e1e56b48c39059bc48d9f2396e: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:51.009919 13244 log.cc:824] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:51.009908 13245 log.cc:824] T a85a23f84dbf477c9287c800182af22f P 5228ff484b824a0ebbc1041d70f9818d: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:51.012097 13249 tablet_bootstrap.cc:492] T a85a23f84dbf477c9287c800182af22f P 954e02e1e56b48c39059bc48d9f2396e: No bootstrap required, opened a new log
I20260812 11:13:51.012142 13244 tablet_bootstrap.cc:492] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d: No bootstrap required, opened a new log
I20260812 11:13:51.012142 13245 tablet_bootstrap.cc:492] T a85a23f84dbf477c9287c800182af22f P 5228ff484b824a0ebbc1041d70f9818d: No bootstrap required, opened a new log
I20260812 11:13:51.012336 13249 ts_tablet_manager.cc:1397] T a85a23f84dbf477c9287c800182af22f P 954e02e1e56b48c39059bc48d9f2396e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 11:13:51.012334 13244 ts_tablet_manager.cc:1397] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 11:13:51.012334 13245 ts_tablet_manager.cc:1397] T a85a23f84dbf477c9287c800182af22f P 5228ff484b824a0ebbc1041d70f9818d: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 11:13:51.016490 13249 raft_consensus.cc:348] T a85a23f84dbf477c9287c800182af22f P 954e02e1e56b48c39059bc48d9f2396e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af6dd25904349b7ac506f69091cef5d" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39787 } } peers { permanent_uuid: "954e02e1e56b48c39059bc48d9f2396e" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 44741 } } peers { permanent_uuid: "5228ff484b824a0ebbc1041d70f9818d" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 40373 } }
I20260812 11:13:51.016773 13249 raft_consensus.cc:374] T a85a23f84dbf477c9287c800182af22f P 954e02e1e56b48c39059bc48d9f2396e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:51.016853 13249 raft_consensus.cc:729] T a85a23f84dbf477c9287c800182af22f P 954e02e1e56b48c39059bc48d9f2396e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 954e02e1e56b48c39059bc48d9f2396e, State: Initialized, Role: FOLLOWER
I20260812 11:13:51.017338 13249 consensus_queue.cc:260] T a85a23f84dbf477c9287c800182af22f P 954e02e1e56b48c39059bc48d9f2396e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af6dd25904349b7ac506f69091cef5d" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39787 } } peers { permanent_uuid: "954e02e1e56b48c39059bc48d9f2396e" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 44741 } } peers { permanent_uuid: "5228ff484b824a0ebbc1041d70f9818d" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 40373 } }
I20260812 11:13:51.017287 13244 raft_consensus.cc:348] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af6dd25904349b7ac506f69091cef5d" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39787 } } peers { permanent_uuid: "954e02e1e56b48c39059bc48d9f2396e" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 44741 } } peers { permanent_uuid: "5228ff484b824a0ebbc1041d70f9818d" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 40373 } }
I20260812 11:13:51.017292 13245 raft_consensus.cc:348] T a85a23f84dbf477c9287c800182af22f P 5228ff484b824a0ebbc1041d70f9818d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af6dd25904349b7ac506f69091cef5d" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39787 } } peers { permanent_uuid: "954e02e1e56b48c39059bc48d9f2396e" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 44741 } } peers { permanent_uuid: "5228ff484b824a0ebbc1041d70f9818d" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 40373 } }
I20260812 11:13:51.017522 13245 raft_consensus.cc:374] T a85a23f84dbf477c9287c800182af22f P 5228ff484b824a0ebbc1041d70f9818d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:51.017522 13244 raft_consensus.cc:374] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:51.017596 13244 raft_consensus.cc:729] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2af6dd25904349b7ac506f69091cef5d, State: Initialized, Role: FOLLOWER
I20260812 11:13:51.017612 13245 raft_consensus.cc:729] T a85a23f84dbf477c9287c800182af22f P 5228ff484b824a0ebbc1041d70f9818d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5228ff484b824a0ebbc1041d70f9818d, State: Initialized, Role: FOLLOWER
I20260812 11:13:51.018046 13022 heartbeater.cc:503] Master 127.17.44.254:33129 was elected leader, sending a full tablet report...
I20260812 11:13:51.018013 13244 consensus_queue.cc:260] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af6dd25904349b7ac506f69091cef5d" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39787 } } peers { permanent_uuid: "954e02e1e56b48c39059bc48d9f2396e" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 44741 } } peers { permanent_uuid: "5228ff484b824a0ebbc1041d70f9818d" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 40373 } }
I20260812 11:13:51.018033 13245 consensus_queue.cc:260] T a85a23f84dbf477c9287c800182af22f P 5228ff484b824a0ebbc1041d70f9818d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af6dd25904349b7ac506f69091cef5d" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39787 } } peers { permanent_uuid: "954e02e1e56b48c39059bc48d9f2396e" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 44741 } } peers { permanent_uuid: "5228ff484b824a0ebbc1041d70f9818d" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 40373 } }
I20260812 11:13:51.018648 12809 heartbeater.cc:503] Master 127.17.44.254:33129 was elected leader, sending a full tablet report...
I20260812 11:13:51.018999 13244 ts_tablet_manager.cc:1428] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 11:13:51.018999 13245 ts_tablet_manager.cc:1428] T a85a23f84dbf477c9287c800182af22f P 5228ff484b824a0ebbc1041d70f9818d: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 11:13:51.019294 13222 heartbeater.cc:503] Master 127.17.44.254:33129 was elected leader, sending a full tablet report...
I20260812 11:13:51.020327 13249 ts_tablet_manager.cc:1428] T a85a23f84dbf477c9287c800182af22f P 954e02e1e56b48c39059bc48d9f2396e: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
W20260812 11:13:51.023861 12811 tablet.cc:2369] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:13:51.188766 13223 tablet.cc:2369] T a85a23f84dbf477c9287c800182af22f P 5228ff484b824a0ebbc1041d70f9818d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:13:51.242444 13024 tablet.cc:2369] T a85a23f84dbf477c9287c800182af22f P 954e02e1e56b48c39059bc48d9f2396e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:13:51.325173 13256 raft_consensus.cc:482] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:13:51.325443 13256 raft_consensus.cc:504] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af6dd25904349b7ac506f69091cef5d" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39787 } } peers { permanent_uuid: "954e02e1e56b48c39059bc48d9f2396e" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 44741 } } peers { permanent_uuid: "5228ff484b824a0ebbc1041d70f9818d" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 40373 } }
I20260812 11:13:51.326694 13256 leader_election.cc:288] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 954e02e1e56b48c39059bc48d9f2396e (127.17.44.194:44741), 5228ff484b824a0ebbc1041d70f9818d (127.17.44.195:40373)
I20260812 11:13:51.331122 13243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:51.327310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:60169 (local address 127.17.44.195:40373)
0812 11:13:51.327444 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:51.327449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:51.327564 (+ 115us) server_negotiation.cc:407] Connection header received
0812 11:13:51.327749 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:51.327752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:51.327807 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:51.327892 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:51.328310 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:51.328833 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:51.330098 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:51.330532 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:51.330668 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:51.330723 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:51.330912 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 11:13:51.331010 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 11:13:51.331141 13261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:51.327087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:40373 (local address 127.17.44.193:60169)
0812 11:13:51.327445 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:51.327462 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:51.327632 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:51.327913 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:51.327916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:51.327933 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:51.328078 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:51.328084 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:51.329034 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:51.329041 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:51.329959 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:51.329969 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:51.330666 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:51.330670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:51.330773 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:51.330799 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 11:13:51.330894 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 11:13:51.330982 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":32,"thread_start_us":102,"threads_started":1}
I20260812 11:13:51.331280 13240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:51.327226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:54469 (local address 127.17.44.194:44741)
0812 11:13:51.327440 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:51.327445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:51.327601 (+ 156us) server_negotiation.cc:407] Connection header received
0812 11:13:51.327885 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:51.327888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:51.327932 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:51.328020 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:51.328319 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:51.329014 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:51.330168 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:51.330589 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:51.330743 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:51.330802 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:51.331051 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 11:13:51.331125 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260812 11:13:51.331562 13259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:51.327087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:44741 (local address 127.17.44.193:54469)
0812 11:13:51.327506 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:51.327518 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:51.327733 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:51.328035 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:51.328037 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:51.328055 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:51.328193 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:51.328207 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:51.329169 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:51.329176 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:51.330021 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:51.330031 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:51.330744 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:51.330748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:51.330854 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:51.330884 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 11:13:51.331029 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 11:13:51.331104 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":33,"thread_start_us":107,"threads_started":1}
I20260812 11:13:51.332098 13127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a85a23f84dbf477c9287c800182af22f" candidate_uuid: "2af6dd25904349b7ac506f69091cef5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5228ff484b824a0ebbc1041d70f9818d" is_pre_election: true
I20260812 11:13:51.332183 12943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a85a23f84dbf477c9287c800182af22f" candidate_uuid: "2af6dd25904349b7ac506f69091cef5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "954e02e1e56b48c39059bc48d9f2396e" is_pre_election: true
I20260812 11:13:51.332437 13127 raft_consensus.cc:2454] T a85a23f84dbf477c9287c800182af22f P 5228ff484b824a0ebbc1041d70f9818d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2af6dd25904349b7ac506f69091cef5d in term 0.
I20260812 11:13:51.332479 12943 raft_consensus.cc:2454] T a85a23f84dbf477c9287c800182af22f P 954e02e1e56b48c39059bc48d9f2396e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2af6dd25904349b7ac506f69091cef5d in term 0.
I20260812 11:13:51.332994 12640 leader_election.cc:302] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2af6dd25904349b7ac506f69091cef5d, 5228ff484b824a0ebbc1041d70f9818d, 954e02e1e56b48c39059bc48d9f2396e; no voters:
I20260812 11:13:51.333319 13256 raft_consensus.cc:2781] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:13:51.333398 13256 raft_consensus.cc:482] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:13:51.333431 13256 raft_consensus.cc:3037] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:51.334733 13256 raft_consensus.cc:504] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af6dd25904349b7ac506f69091cef5d" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39787 } } peers { permanent_uuid: "954e02e1e56b48c39059bc48d9f2396e" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 44741 } } peers { permanent_uuid: "5228ff484b824a0ebbc1041d70f9818d" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 40373 } }
I20260812 11:13:51.335162 13256 leader_election.cc:288] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d [CANDIDATE]: Term 1 election: Requested vote from peers 954e02e1e56b48c39059bc48d9f2396e (127.17.44.194:44741), 5228ff484b824a0ebbc1041d70f9818d (127.17.44.195:40373)
I20260812 11:13:51.335585 13127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a85a23f84dbf477c9287c800182af22f" candidate_uuid: "2af6dd25904349b7ac506f69091cef5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5228ff484b824a0ebbc1041d70f9818d"
I20260812 11:13:51.335728 13127 raft_consensus.cc:3037] T a85a23f84dbf477c9287c800182af22f P 5228ff484b824a0ebbc1041d70f9818d [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:51.335774 12943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a85a23f84dbf477c9287c800182af22f" candidate_uuid: "2af6dd25904349b7ac506f69091cef5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "954e02e1e56b48c39059bc48d9f2396e"
I20260812 11:13:51.335892 12943 raft_consensus.cc:3037] T a85a23f84dbf477c9287c800182af22f P 954e02e1e56b48c39059bc48d9f2396e [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:51.337054 13127 raft_consensus.cc:2454] T a85a23f84dbf477c9287c800182af22f P 5228ff484b824a0ebbc1041d70f9818d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2af6dd25904349b7ac506f69091cef5d in term 1.
I20260812 11:13:51.337184 12943 raft_consensus.cc:2454] T a85a23f84dbf477c9287c800182af22f P 954e02e1e56b48c39059bc48d9f2396e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2af6dd25904349b7ac506f69091cef5d in term 1.
I20260812 11:13:51.337520 12638 leader_election.cc:302] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2af6dd25904349b7ac506f69091cef5d, 954e02e1e56b48c39059bc48d9f2396e; no voters:
I20260812 11:13:51.337708 13256 raft_consensus.cc:2781] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:13:51.337947 13256 raft_consensus.cc:686] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d [term 1 LEADER]: Becoming Leader. State: Replica: 2af6dd25904349b7ac506f69091cef5d, State: Running, Role: LEADER
I20260812 11:13:51.338274 13256 consensus_queue.cc:237] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af6dd25904349b7ac506f69091cef5d" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39787 } } peers { permanent_uuid: "954e02e1e56b48c39059bc48d9f2396e" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 44741 } } peers { permanent_uuid: "5228ff484b824a0ebbc1041d70f9818d" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 40373 } }
I20260812 11:13:51.341445 12533 catalog_manager.cc:5495] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2af6dd25904349b7ac506f69091cef5d (127.17.44.193). New cstate: current_term: 1 leader_uuid: "2af6dd25904349b7ac506f69091cef5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2af6dd25904349b7ac506f69091cef5d" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 39787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "954e02e1e56b48c39059bc48d9f2396e" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 44741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5228ff484b824a0ebbc1041d70f9818d" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 40373 } health_report { overall_health: UNKNOWN } } }
I20260812 11:13:51.357945 13235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:51.354256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42290 (local address 127.17.44.193:39787)
0812 11:13:51.354488 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:51.354492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:51.354504 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:13:51.354567 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:51.354569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:51.354618 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:51.354707 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:13:51.355126 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:51.355690 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:51.356553 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:51.356771 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:51.356861 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:51.357149 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:13:51.357557 (+ 408us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:13:51.357651 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:51.357796 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 11:13:51.357844 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260812 11:13:51.364030 13127 raft_consensus.cc:1261] T a85a23f84dbf477c9287c800182af22f P 5228ff484b824a0ebbc1041d70f9818d [term 1 FOLLOWER]: Refusing update from remote peer 2af6dd25904349b7ac506f69091cef5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:13:51.364030 12943 raft_consensus.cc:1261] T a85a23f84dbf477c9287c800182af22f P 954e02e1e56b48c39059bc48d9f2396e [term 1 FOLLOWER]: Refusing update from remote peer 2af6dd25904349b7ac506f69091cef5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:13:51.364749 13263 consensus_queue.cc:1035] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5228ff484b824a0ebbc1041d70f9818d" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 40373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:13:51.364924 13256 consensus_queue.cc:1035] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "954e02e1e56b48c39059bc48d9f2396e" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 44741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:13:51.374620 13271 mvcc.cc:205] Tried to move back new op lower bound from 7317640115659276288 to 7317640115563954176. Current Snapshot: MvccSnapshot[applied={T|T < 7317640115659276288}]
I20260812 11:13:51.377241 13268 mvcc.cc:205] Tried to move back new op lower bound from 7317640115659276288 to 7317640115563954176. Current Snapshot: MvccSnapshot[applied={T|T < 7317640115659276288 or (T in {7317640115659276288})}]
I20260812 11:13:51.382210 12533 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57016:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:13:51.382504 12533 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57016:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:13:51.385829 12533 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0975ff41a46349ec8809cabb2fb5dab7: Sending DeleteTablet for 3 replicas of tablet a85a23f84dbf477c9287c800182af22f
I20260812 11:13:51.386723 12709 tablet_service.cc:1514] Processing DeleteTablet for tablet a85a23f84dbf477c9287c800182af22f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:13:51 UTC) from {username='slave'} at 127.0.0.1:42276
I20260812 11:13:51.386807 13097 tablet_service.cc:1514] Processing DeleteTablet for tablet a85a23f84dbf477c9287c800182af22f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:13:51 UTC) from {username='slave'} at 127.0.0.1:42440
I20260812 11:13:51.386909 12911 tablet_service.cc:1514] Processing DeleteTablet for tablet a85a23f84dbf477c9287c800182af22f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:13:51 UTC) from {username='slave'} at 127.0.0.1:48178
I20260812 11:13:51.387315 13283 tablet_replica.cc:330] T a85a23f84dbf477c9287c800182af22f P 2af6dd25904349b7ac506f69091cef5d: stopping tablet replica
I20260812 11:13:51.387357 13285 tablet_replica.cc:330] T a85a23f84dbf477c9287c800182af22f P 954e02e1e56b48c39059bc48d9f2396e: stopping tablet replica
I20260812 11:13:51.387555 13284 tablet_replica.cc:330] T a85a23f84dbf477c9287c800182af22f P 5228ff484b824a0ebbc1041d70f9818d: stopping tablet replica
I20260812 11:13:51.387728 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 12615
I20260812 11:13:51.388001 13285 raft_consensus.cc:2229] T a85a23f84dbf477c9287c800182af22f P 954e02e1e56b48c39059bc48d9f2396e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:13:51.388319 13285 raft_consensus.cc:2258] T a85a23f84dbf477c9287c800182af22f P 954e02e1e56b48c39059bc48d9f2396e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:13:51.388715 13284 raft_consensus.cc:2229] T a85a23f84dbf477c9287c800182af22f P 5228ff484b824a0ebbc1041d70f9818d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:13:51.388952 13284 raft_consensus.cc:2258] T a85a23f84dbf477c9287c800182af22f P 5228ff484b824a0ebbc1041d70f9818d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:13:51.393567 13284 ts_tablet_manager.cc:1905] T a85a23f84dbf477c9287c800182af22f P 5228ff484b824a0ebbc1041d70f9818d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:13:51.394045 13285 ts_tablet_manager.cc:1905] T a85a23f84dbf477c9287c800182af22f P 954e02e1e56b48c39059bc48d9f2396e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:13:51.396317 13285 ts_tablet_manager.cc:1918] T a85a23f84dbf477c9287c800182af22f P 954e02e1e56b48c39059bc48d9f2396e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:13:51.396430 13285 log.cc:1196] T a85a23f84dbf477c9287c800182af22f P 954e02e1e56b48c39059bc48d9f2396e: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/a85a23f84dbf477c9287c800182af22f
I20260812 11:13:51.396771 13285 ts_tablet_manager.cc:1939] T a85a23f84dbf477c9287c800182af22f P 954e02e1e56b48c39059bc48d9f2396e: Deleting consensus metadata
I20260812 11:13:51.397096 13284 ts_tablet_manager.cc:1918] T a85a23f84dbf477c9287c800182af22f P 5228ff484b824a0ebbc1041d70f9818d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:13:51.397200 13284 log.cc:1196] T a85a23f84dbf477c9287c800182af22f P 5228ff484b824a0ebbc1041d70f9818d: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/a85a23f84dbf477c9287c800182af22f
I20260812 11:13:51.397539 13284 ts_tablet_manager.cc:1939] T a85a23f84dbf477c9287c800182af22f P 5228ff484b824a0ebbc1041d70f9818d: Deleting consensus metadata
I20260812 11:13:51.398193 12500 catalog_manager.cc:4841] TS 954e02e1e56b48c39059bc48d9f2396e (127.17.44.194:44741): tablet a85a23f84dbf477c9287c800182af22f (table test-table [id=2cffd59b65a34ff8acfe185425fd7233]) successfully deleted
I20260812 11:13:51.398676 12502 catalog_manager.cc:4841] TS 5228ff484b824a0ebbc1041d70f9818d (127.17.44.195:40373): tablet a85a23f84dbf477c9287c800182af22f (table test-table [id=2cffd59b65a34ff8acfe185425fd7233]) successfully deleted
W20260812 11:13:51.401523 12502 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.44.193:39787 (error 108)
I20260812 11:13:51.401895 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 12814
I20260812 11:13:51.406101 13239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:51.405603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:39787 (local address 127.0.0.1:42292)
0812 11:13:51.405887 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:51.405987 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.193:39787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260812 11:13:51.406437 12502 catalog_manager.cc:4584] TS 2af6dd25904349b7ac506f69091cef5d (127.17.44.193:39787): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a85a23f84dbf477c9287c800182af22f: Network error: Client connection negotiation failed: client connection to 127.17.44.193:39787: connect: Connection refused (error 111)
I20260812 11:13:51.415529 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 13026
I20260812 11:13:51.424643 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 12476
2026-08-12T11:13:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1404 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:13:51 dist-test-slave-drl0 krb5kdc[13292](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:13:51 dist-test-slave-drl0 krb5kdc[13292](info): set up 2 sockets
Aug 12 11:13:51 dist-test-slave-drl0 krb5kdc[13292](info): commencing operation
krb5kdc: starting...
W20260812 11:13:52.704406 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:13:52 dist-test-slave-drl0 krb5kdc[13292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:13:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:13:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:13:52.842249 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:41899
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:35633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:41899
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:13:52.967619 13321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:52.967955 13321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:52.968039 13321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:52.972633 13321 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:13:52.972733 13321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:52.972774 13321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:52.972836 13321 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:13:52.972877 13321 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:13:52.979501 13321 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:35633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:41899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:41899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:52.980593 13321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:52.982038 13321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:52.991323 13331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:52.991458 13327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:52.991338 13329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:52.992051 13321 server_base.cc:1034] running on GCE node
I20260812 11:13:52.992646 13321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:52.993749 13321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:52.995007 13321 hybrid_clock.cc:648] HybridClock initialized: now 1786533232994989 us; error 33 us; skew 500 ppm
Aug 12 11:13:52 dist-test-slave-drl0 krb5kdc[13292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533232, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:13:52.998798 13321 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:13:52.999872 13321 webserver.cc:460] Webserver started at http://127.17.44.254:37891/ using document root <none> and password file <none>
I20260812 11:13:53.000506 13321 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:53.000576 13321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:53.000815 13321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:13:53.002856 13321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "4fdec254e4c34b10b9333bca3d5e8a45"
format_stamp: "Formatted at 2026-08-12 11:13:52 on dist-test-slave-drl0"
server_key: "977c53234d66643f49fc88e33eb535db"
server_key_iv: "f9ac1d8a373bb1349e9bf163dfaf77f9"
server_key_version: "encryptionkey@0"
I20260812 11:13:53.003856 13321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "4fdec254e4c34b10b9333bca3d5e8a45"
format_stamp: "Formatted at 2026-08-12 11:13:52 on dist-test-slave-drl0"
server_key: "977c53234d66643f49fc88e33eb535db"
server_key_iv: "f9ac1d8a373bb1349e9bf163dfaf77f9"
server_key_version: "encryptionkey@0"
I20260812 11:13:53.008567 13321 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260812 11:13:53.012356 13340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:53.013984 13321 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260812 11:13:53.014161 13321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "4fdec254e4c34b10b9333bca3d5e8a45"
format_stamp: "Formatted at 2026-08-12 11:13:52 on dist-test-slave-drl0"
server_key: "977c53234d66643f49fc88e33eb535db"
server_key_iv: "f9ac1d8a373bb1349e9bf163dfaf77f9"
server_key_version: "encryptionkey@0"
I20260812 11:13:53.014290 13321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:53.044124 13321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:53.047562 13321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:53.047772 13321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:53.055217 13321 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:41899
I20260812 11:13:53.055239 13410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:41899 every 8 connection(s)
I20260812 11:13:53.056229 13321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:13:53.060132 13412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:53.061770 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 13321
I20260812 11:13:53.061962 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:13:53.062279 17587 external_mini_cluster.cc:1428] Setting key bd567909674c4e1563d6a2c9149f1ff1
I20260812 11:13:53.067453 13412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fdec254e4c34b10b9333bca3d5e8a45: Bootstrap starting.
I20260812 11:13:53.069674 13412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fdec254e4c34b10b9333bca3d5e8a45: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:53.070429 13412 log.cc:824] T 00000000000000000000000000000000 P 4fdec254e4c34b10b9333bca3d5e8a45: Log is configured to *not* fsync() on all Append() calls
Aug 12 11:13:53 dist-test-slave-drl0 krb5kdc[13292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:13:53.073200 13412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fdec254e4c34b10b9333bca3d5e8a45: No bootstrap required, opened a new log
I20260812 11:13:53.078259 13417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:53.063989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54778 (local address 127.17.44.254:41899)
0812 11:13:53.064475 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:53.064492 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:53.064531 (+ 39us) server_negotiation.cc:407] Connection header received
0812 11:13:53.065261 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:53.065280 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:53.065631 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:53.066018 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:53.066980 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.068043 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.069317 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.069840 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.070110 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:53.073648 (+ 3538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:53.073653 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:53.073665 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:13:53.073692 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:53.075487 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:53.075941 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:53.075946 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:53.075951 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:53.076058 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:53.076354 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:53.076357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:53.076359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:53.076742 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:53.076988 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:53.077229 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 11:13:53.077521 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":147,"threads_started":1}
I20260812 11:13:53.078358 13412 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fdec254e4c34b10b9333bca3d5e8a45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fdec254e4c34b10b9333bca3d5e8a45" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 41899 } }
I20260812 11:13:53.078768 13412 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fdec254e4c34b10b9333bca3d5e8a45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:53.078858 13412 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fdec254e4c34b10b9333bca3d5e8a45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fdec254e4c34b10b9333bca3d5e8a45, State: Initialized, Role: FOLLOWER
I20260812 11:13:53.079459 13412 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fdec254e4c34b10b9333bca3d5e8a45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fdec254e4c34b10b9333bca3d5e8a45" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 41899 } }
I20260812 11:13:53.079640 13412 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fdec254e4c34b10b9333bca3d5e8a45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:13:53.079738 13412 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fdec254e4c34b10b9333bca3d5e8a45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:13:53.079870 13412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fdec254e4c34b10b9333bca3d5e8a45 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:53.081434 13412 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fdec254e4c34b10b9333bca3d5e8a45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fdec254e4c34b10b9333bca3d5e8a45" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 41899 } }
I20260812 11:13:53.081871 13412 leader_election.cc:302] T 00000000000000000000000000000000 P 4fdec254e4c34b10b9333bca3d5e8a45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fdec254e4c34b10b9333bca3d5e8a45; no voters:
I20260812 11:13:53.082255 13412 leader_election.cc:288] T 00000000000000000000000000000000 P 4fdec254e4c34b10b9333bca3d5e8a45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:13:53.082531 13422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fdec254e4c34b10b9333bca3d5e8a45 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:13:53.082952 13422 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fdec254e4c34b10b9333bca3d5e8a45 [term 1 LEADER]: Becoming Leader. State: Replica: 4fdec254e4c34b10b9333bca3d5e8a45, State: Running, Role: LEADER
I20260812 11:13:53.083300 13422 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fdec254e4c34b10b9333bca3d5e8a45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fdec254e4c34b10b9333bca3d5e8a45" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 41899 } }
I20260812 11:13:53.083644 13412 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fdec254e4c34b10b9333bca3d5e8a45 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:13:53.085126 13425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fdec254e4c34b10b9333bca3d5e8a45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fdec254e4c34b10b9333bca3d5e8a45. Latest consensus state: current_term: 1 leader_uuid: "4fdec254e4c34b10b9333bca3d5e8a45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fdec254e4c34b10b9333bca3d5e8a45" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 41899 } } }
I20260812 11:13:53.085119 13424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fdec254e4c34b10b9333bca3d5e8a45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fdec254e4c34b10b9333bca3d5e8a45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fdec254e4c34b10b9333bca3d5e8a45" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 41899 } } }
I20260812 11:13:53.085318 13425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fdec254e4c34b10b9333bca3d5e8a45 [sys.catalog]: This master's current role is: LEADER
I20260812 11:13:53.085628 13424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fdec254e4c34b10b9333bca3d5e8a45 [sys.catalog]: This master's current role is: LEADER
I20260812 11:13:53.085794 13433 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:13:53.088965 13433 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:13:53.097230 13433 catalog_manager.cc:1347] Generated new cluster ID: 4a6e9116a0654354b5062712905f9de7
I20260812 11:13:53.097365 13433 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:13:53.112165 13433 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:13:53.113229 13433 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:13:53.121546 13433 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fdec254e4c34b10b9333bca3d5e8a45: Generated new TSK 0
I20260812 11:13:53.122262 13433 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
I20260812 11:13:53.164206 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:41899
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:35633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:13:53.284766 13447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:53.285058 13447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:53.285364 13447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:53.291287 13447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:53.291395 13447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:53.291504 13447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:13:53.298199 13447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:35633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:41899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:53.299435 13447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:53.300509 13447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:53.311564 13462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:53.311697 13460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:53.311640 13447 server_base.cc:1034] running on GCE node
W20260812 11:13:53.311570 13465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:53.312204 13447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:53.312839 13447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:53.314031 13447 hybrid_clock.cc:648] HybridClock initialized: now 1786533233314014 us; error 36 us; skew 500 ppm
Aug 12 11:13:53 dist-test-slave-drl0 krb5kdc[13292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533233, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:13:53.317097 13447 init.cc:377] Logged in from keytab as kudu/127.17.44.193@KRBTEST.COM (short username kudu)
I20260812 11:13:53.318186 13447 webserver.cc:460] Webserver started at http://127.17.44.193:39759/ using document root <none> and password file <none>
I20260812 11:13:53.318778 13447 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:53.318872 13447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:53.319134 13447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:13:53.321539 13447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "c5db80c089ef445686ab9b90db2d144c"
format_stamp: "Formatted at 2026-08-12 11:13:53 on dist-test-slave-drl0"
server_key: "62809bb3fd86f2285f54f7d6b07aee37"
server_key_iv: "962d7f46ae11b3530fad6c15a7ad4599"
server_key_version: "encryptionkey@0"
I20260812 11:13:53.322146 13447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "c5db80c089ef445686ab9b90db2d144c"
format_stamp: "Formatted at 2026-08-12 11:13:53 on dist-test-slave-drl0"
server_key: "62809bb3fd86f2285f54f7d6b07aee37"
server_key_iv: "962d7f46ae11b3530fad6c15a7ad4599"
server_key_version: "encryptionkey@0"
I20260812 11:13:53.326907 13447 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 11:13:53.329521 13475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:53.331347 13447 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 11:13:53.331499 13447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "c5db80c089ef445686ab9b90db2d144c"
format_stamp: "Formatted at 2026-08-12 11:13:53 on dist-test-slave-drl0"
server_key: "62809bb3fd86f2285f54f7d6b07aee37"
server_key_iv: "962d7f46ae11b3530fad6c15a7ad4599"
server_key_version: "encryptionkey@0"
I20260812 11:13:53.331614 13447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:53.358498 13447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:53.361907 13447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:53.362106 13447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:53.362711 13447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:13:53.363595 13447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:13:53.363667 13447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:53.363739 13447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:13:53.363793 13447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:53.374673 13447 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:34177
I20260812 11:13:53.374701 13652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:34177 every 8 connection(s)
I20260812 11:13:53.376044 13447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:13:53.381704 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 13447
I20260812 11:13:53.381878 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:13:53.382205 17587 external_mini_cluster.cc:1428] Setting key 48aab199d7acd802757eddfc9a50c41d
Aug 12 11:13:53 dist-test-slave-drl0 krb5kdc[13292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533233, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:13:53.392926 13417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:53.378306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:37759 (local address 127.17.44.254:41899)
0812 11:13:53.378490 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:53.378496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:53.379547 (+ 1051us) server_negotiation.cc:407] Connection header received
0812 11:13:53.380848 (+ 1301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:53.380851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:53.380905 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:53.381000 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:53.382534 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.383304 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.384770 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.385083 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.385234 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:53.388368 (+ 3134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:53.388372 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:53.388374 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:13:53.388396 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:53.390280 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:53.390831 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:53.390836 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:53.390838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:53.390969 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:53.391333 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:53.391336 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:53.391338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:53.391525 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:53.391808 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:53.392538 (+ 730us) server_negotiation.cc:299] Negotiation successful
0812 11:13:53.392724 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 11:13:53.393987 13655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:53.378639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:41899 (local address 127.17.44.193:37759)
0812 11:13:53.379387 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:53.379430 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:53.380635 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:53.381158 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:53.381164 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:53.381641 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:13:53.382324 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:53.382345 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.383462 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.383464 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:53.384643 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:53.384655 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.385281 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.385288 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:53.385545 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:53.386546 (+ 1001us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:13:53.386573 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:13:53.388191 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:13:53.390429 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:53.390434 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:53.390449 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:53.390711 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:53.391075 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:53.391078 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:53.391080 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:53.391222 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:53.391808 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:13:53.391820 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:13:53.392227 (+ 407us) client_negotiation.cc:769] Sending connection context
0812 11:13:53.392479 (+ 252us) client_negotiation.cc:240] Negotiation successful
0812 11:13:53.392935 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":208,"threads_started":1}
WARNING: no policy specified for kudu/127.17.44.194@KRBTEST.COM; defaulting to no policy
I20260812 11:13:53.395596 13653 heartbeater.cc:347] Connected to a master server at 127.17.44.254:41899
I20260812 11:13:53.395846 13653 heartbeater.cc:464] Registering TS with master...
I20260812 11:13:53.396456 13653 heartbeater.cc:511] Master 127.17.44.254:41899 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.194@KRBTEST.COM" created.
I20260812 11:13:53.398303 13362 ts_manager.cc:194] Registered new tserver with Master: c5db80c089ef445686ab9b90db2d144c (127.17.44.193:34177)
I20260812 11:13:53.399561 13362 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.193@KRBTEST.COM'} at 127.17.44.193:37759
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
I20260812 11:13:53.418572 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:41899
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:35633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:13:53.543020 13663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:53.543284 13663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:53.543327 13663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:53.547602 13663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:53.547662 13663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:53.547719 13663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:13:53.553115 13663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:35633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:41899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:53.555111 13663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:53.556572 13663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:53.567186 13678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:53.567237 13675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:53.567188 13673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:53.567488 13663 server_base.cc:1034] running on GCE node
I20260812 11:13:53.567903 13663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:53.568574 13663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:53.569770 13663 hybrid_clock.cc:648] HybridClock initialized: now 1786533233569738 us; error 42 us; skew 500 ppm
Aug 12 11:13:53 dist-test-slave-drl0 krb5kdc[13292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533233, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:13:53.572736 13663 init.cc:377] Logged in from keytab as kudu/127.17.44.194@KRBTEST.COM (short username kudu)
I20260812 11:13:53.573834 13663 webserver.cc:460] Webserver started at http://127.17.44.194:43267/ using document root <none> and password file <none>
I20260812 11:13:53.574749 13663 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:53.574865 13663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:53.575201 13663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:13:53.578599 13663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "bd5ca16bb6a1466d8cdd7c3c4ee4f511"
format_stamp: "Formatted at 2026-08-12 11:13:53 on dist-test-slave-drl0"
server_key: "e67923dbba47b6f2c62baee80ebb8744"
server_key_iv: "7e50afea5641bf9ded15e0ec9348fa36"
server_key_version: "encryptionkey@0"
I20260812 11:13:53.579346 13663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "bd5ca16bb6a1466d8cdd7c3c4ee4f511"
format_stamp: "Formatted at 2026-08-12 11:13:53 on dist-test-slave-drl0"
server_key: "e67923dbba47b6f2c62baee80ebb8744"
server_key_iv: "7e50afea5641bf9ded15e0ec9348fa36"
server_key_version: "encryptionkey@0"
I20260812 11:13:53.583783 13663 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 11:13:53.586627 13688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:53.588132 13663 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 11:13:53.588267 13663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "bd5ca16bb6a1466d8cdd7c3c4ee4f511"
format_stamp: "Formatted at 2026-08-12 11:13:53 on dist-test-slave-drl0"
server_key: "e67923dbba47b6f2c62baee80ebb8744"
server_key_iv: "7e50afea5641bf9ded15e0ec9348fa36"
server_key_version: "encryptionkey@0"
I20260812 11:13:53.588397 13663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:53.625401 13663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:53.628957 13663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:53.629172 13663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:53.629806 13663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:13:53.630681 13663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:13:53.630755 13663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:53.630826 13663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:13:53.630887 13663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:53.640681 13663 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:45233
I20260812 11:13:53.640715 13862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:45233 every 8 connection(s)
I20260812 11:13:53.641978 13663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:13:53.651503 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 13663
I20260812 11:13:53.651666 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:13:53.651999 17587 external_mini_cluster.cc:1428] Setting key cc5309f1906d9cd8ec0184c22491ad6e
Aug 12 11:13:53 dist-test-slave-drl0 krb5kdc[13292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533233, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:13:53.657382 13417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:53.644208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:44575 (local address 127.17.44.254:41899)
0812 11:13:53.644384 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:53.644388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:53.645310 (+ 922us) server_negotiation.cc:407] Connection header received
0812 11:13:53.646669 (+ 1359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:53.646672 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:53.646732 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:53.646828 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:53.648032 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.648570 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.649717 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.649948 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.650062 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:53.653175 (+ 3113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:53.653179 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:53.653181 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:13:53.653206 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:53.654652 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:53.655382 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:53.655385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:53.655386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:53.655483 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:53.655955 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:53.655962 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:53.655966 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:53.656271 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:53.656397 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:53.656968 (+ 571us) server_negotiation.cc:299] Negotiation successful
0812 11:13:53.657122 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 11:13:53.658051 13865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:53.644666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:41899 (local address 127.17.44.194:44575)
0812 11:13:53.645141 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:53.645185 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:53.646454 (+ 1269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:53.647053 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:53.647060 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:53.647413 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:13:53.647878 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:53.647890 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.648700 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.648704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:53.649582 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:53.649592 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.650102 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.650106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:53.650285 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:53.651132 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:13:53.651264 (+ 132us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:13:53.653002 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:13:53.654789 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:53.654798 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:53.654816 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:53.655232 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:53.655592 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:53.655598 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:53.655602 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:53.655791 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:53.656441 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:13:53.656446 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:13:53.656681 (+ 235us) client_negotiation.cc:769] Sending connection context
0812 11:13:53.656852 (+ 171us) client_negotiation.cc:240] Negotiation successful
0812 11:13:53.657107 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":110,"threads_started":1}
I20260812 11:13:53.659428 13863 heartbeater.cc:347] Connected to a master server at 127.17.44.254:41899
I20260812 11:13:53.659791 13863 heartbeater.cc:464] Registering TS with master...
I20260812 11:13:53.660475 13863 heartbeater.cc:511] Master 127.17.44.254:41899 requested a full tablet report, sending...
I20260812 11:13:53.661682 13362 ts_manager.cc:194] Registered new tserver with Master: bd5ca16bb6a1466d8cdd7c3c4ee4f511 (127.17.44.194:45233)
I20260812 11:13:53.662683 13362 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.194@KRBTEST.COM'} at 127.17.44.194:44575
WARNING: no policy specified for kudu/127.17.44.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
I20260812 11:13:53.686545 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:41899
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:35633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:13:53.813854 13871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:53.814105 13871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:53.814147 13871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:53.818640 13871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:53.818703 13871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:53.818787 13871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:13:53.824499 13871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:35633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:41899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:53.825603 13871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:53.826654 13871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:53.836887 13881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:53.836884 13883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:53.836892 13880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:53.837167 13871 server_base.cc:1034] running on GCE node
I20260812 11:13:53.837669 13871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:53.838312 13871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:53.839529 13871 hybrid_clock.cc:648] HybridClock initialized: now 1786533233839494 us; error 54 us; skew 500 ppm
Aug 12 11:13:53 dist-test-slave-drl0 krb5kdc[13292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533233, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:13:53.842502 13871 init.cc:377] Logged in from keytab as kudu/127.17.44.195@KRBTEST.COM (short username kudu)
I20260812 11:13:53.843539 13871 webserver.cc:460] Webserver started at http://127.17.44.195:35401/ using document root <none> and password file <none>
I20260812 11:13:53.844202 13871 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:53.844278 13871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:53.844548 13871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:13:53.847512 13871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "373718738acd43abb245a898c90fa249"
format_stamp: "Formatted at 2026-08-12 11:13:53 on dist-test-slave-drl0"
server_key: "bde85a555e170f1b2283c232c64872db"
server_key_iv: "15b12c5f0e279e68c26182f2ca4dc6f8"
server_key_version: "encryptionkey@0"
I20260812 11:13:53.848134 13871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "373718738acd43abb245a898c90fa249"
format_stamp: "Formatted at 2026-08-12 11:13:53 on dist-test-slave-drl0"
server_key: "bde85a555e170f1b2283c232c64872db"
server_key_iv: "15b12c5f0e279e68c26182f2ca4dc6f8"
server_key_version: "encryptionkey@0"
I20260812 11:13:53.852442 13871 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 11:13:53.855557 13894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:53.857343 13871 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:13:53.857475 13871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "373718738acd43abb245a898c90fa249"
format_stamp: "Formatted at 2026-08-12 11:13:53 on dist-test-slave-drl0"
server_key: "bde85a555e170f1b2283c232c64872db"
server_key_iv: "15b12c5f0e279e68c26182f2ca4dc6f8"
server_key_version: "encryptionkey@0"
I20260812 11:13:53.857641 13871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:53.888705 13871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:53.892402 13871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:53.892673 13871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:53.893388 13871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:13:53.894371 13871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:13:53.894434 13871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:53.894477 13871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:13:53.894493 13871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:53.905025 13871 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:45133
I20260812 11:13:53.905048 14086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:45133 every 8 connection(s)
I20260812 11:13:53.905933 13871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:13:53.914901 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 13871
I20260812 11:13:53.915127 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:13:53.915434 17587 external_mini_cluster.cc:1428] Setting key 97c2707f743d253108a9e818ec6258f1
Aug 12 11:13:53 dist-test-slave-drl0 krb5kdc[13292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533233, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:13:53.921722 13417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:53.908375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:47377 (local address 127.17.44.254:41899)
0812 11:13:53.908515 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:53.908519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:53.909270 (+ 751us) server_negotiation.cc:407] Connection header received
0812 11:13:53.910635 (+ 1365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:53.910638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:53.910696 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:53.910810 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:53.911987 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.912517 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.913645 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.913868 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.913976 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:53.917591 (+ 3615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:53.917595 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:53.917598 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:13:53.917624 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:53.919571 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:53.920204 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:53.920207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:53.920208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:53.920293 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:53.920719 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:53.920722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:53.920723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:53.920905 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:53.921006 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:53.921514 (+ 508us) server_negotiation.cc:299] Negotiation successful
0812 11:13:53.921608 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 11:13:53.922480 14089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:53.908645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:41899 (local address 127.17.44.195:47377)
0812 11:13:53.909118 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:53.909160 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:53.910411 (+ 1251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:53.911032 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:53.911039 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:53.911373 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:13:53.911841 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:53.911854 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.912646 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.912655 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:53.913514 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:53.913524 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.914027 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.914032 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:53.914200 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:53.915062 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:13:53.915113 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:13:53.917366 (+ 2253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:13:53.919712 (+ 2346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:53.919717 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:53.919728 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:53.920082 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:53.920423 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:53.920426 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:53.920427 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:53.920594 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:53.921037 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:13:53.921043 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:13:53.921271 (+ 228us) client_negotiation.cc:769] Sending connection context
0812 11:13:53.921441 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 11:13:53.921682 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":126,"threads_started":1}
I20260812 11:13:53.923619 14087 heartbeater.cc:347] Connected to a master server at 127.17.44.254:41899
I20260812 11:13:53.923827 14087 heartbeater.cc:464] Registering TS with master...
I20260812 11:13:53.924263 14087 heartbeater.cc:511] Master 127.17.44.254:41899 requested a full tablet report, sending...
I20260812 11:13:53.925256 13362 ts_manager.cc:194] Registered new tserver with Master: 373718738acd43abb245a898c90fa249 (127.17.44.195:45133)
I20260812 11:13:53.925793 13362 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.195@KRBTEST.COM'} at 127.17.44.195:47377
I20260812 11:13:53.930576 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:13:53.941474 13417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:53.932741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38504 (local address 127.17.44.254:41899)
0812 11:13:53.932876 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:53.932879 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:53.933193 (+ 314us) server_negotiation.cc:407] Connection header received
0812 11:13:53.933277 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:53.933280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:53.933345 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:53.933471 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:53.934198 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.934865 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.935984 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.936209 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.936337 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:53.937976 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:53.937979 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:53.937981 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:13:53.938002 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:53.939568 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:53.940122 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:53.940124 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:53.940125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:53.940203 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:53.940599 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:53.940601 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:53.940602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:53.940796 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:53.940913 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:53.941263 (+ 350us) server_negotiation.cc:299] Negotiation successful
0812 11:13:53.941348 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 11:13:53.945024 13362 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38504:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:13:53.948211 13362 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:13:53.965864 14114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:53.960185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43970 (local address 127.17.44.193:34177)
0812 11:13:53.961053 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:53.961059 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:53.961080 (+ 21us) server_negotiation.cc:407] Connection header received
0812 11:13:53.961153 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:53.961158 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:53.961321 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:53.961464 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:53.961749 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.962528 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.964298 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.965095 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.965209 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:53.965368 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:53.965571 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 11:13:53.965680 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":767,"thread_start_us":118,"threads_started":1}
I20260812 11:13:53.965855 14109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:53.959640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:34177 (local address 127.0.0.1:43970)
0812 11:13:53.960700 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:53.960713 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:53.960783 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:53.961455 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:53.961458 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:53.961491 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:53.961623 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:53.961629 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.962726 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.962730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:53.964171 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:53.964182 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.965238 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.965242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:53.965357 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:53.965402 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 11:13:53.965523 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 11:13:53.965659 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1004,"mutex_wait_us":8,"spinlock_wait_cycles":1408,"thread_start_us":101,"threads_started":1}
I20260812 11:13:53.966050 14110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:53.959370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:45133 (local address 127.0.0.1:40600)
0812 11:13:53.960288 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:53.960339 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:53.960473 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:53.961180 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:53.961186 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:53.961210 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:53.961348 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:53.961355 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.963159 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.963162 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:53.964522 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:53.964534 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.965654 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.965658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:53.965744 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:53.965767 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 11:13:53.965817 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 11:13:53.965892 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"mutex_wait_us":60,"thread_start_us":81,"threads_started":1}
I20260812 11:13:53.966377 14111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:53.959532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60692 (local address 127.17.44.194:45233)
0812 11:13:53.960008 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:53.960124 (+ 116us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:53.960662 (+ 538us) server_negotiation.cc:407] Connection header received
0812 11:13:53.960895 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:53.960900 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:53.961125 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:53.961246 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:53.961810 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.962996 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.964669 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.965283 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.965362 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:53.965495 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:53.965929 (+ 434us) server_negotiation.cc:299] Negotiation successful
0812 11:13:53.966225 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":108,"threads_started":1}
I20260812 11:13:53.966738 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:53.959386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:45233 (local address 127.0.0.1:60692)
0812 11:13:53.960565 (+ 1179us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:53.960578 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:53.960736 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:53.961463 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:53.961466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:53.961494 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:53.961641 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:53.961647 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.963159 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.963162 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:53.964529 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:53.964535 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.965589 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.965592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:53.965670 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:53.965689 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 11:13:53.965754 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 11:13:53.965809 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1108,"mutex_wait_us":134,"spinlock_wait_cycles":1280,"thread_start_us":127,"threads_started":1}
I20260812 11:13:53.966827 14115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:53.960332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40600 (local address 127.17.44.195:45133)
0812 11:13:53.960659 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:53.960674 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:53.960698 (+ 24us) server_negotiation.cc:407] Connection header received
0812 11:13:53.960779 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:53.960784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:53.961012 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:53.961180 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:53.961551 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.963008 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.964662 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:53.965508 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:53.966387 (+ 879us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:53.966537 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:53.966587 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 11:13:53.966690 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":150,"threads_started":1}
I20260812 11:13:53.969405 13553 tablet_service.cc:1467] Processing CreateTablet for tablet 0d22f81e4955484bbd5f8b96043e71dd (DEFAULT_TABLE table=test-table [id=15ff31efee9b4c04bccdb2b3e8a9d6f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:13:53.969527 13755 tablet_service.cc:1467] Processing CreateTablet for tablet 0d22f81e4955484bbd5f8b96043e71dd (DEFAULT_TABLE table=test-table [id=15ff31efee9b4c04bccdb2b3e8a9d6f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:13:53.969527 13982 tablet_service.cc:1467] Processing CreateTablet for tablet 0d22f81e4955484bbd5f8b96043e71dd (DEFAULT_TABLE table=test-table [id=15ff31efee9b4c04bccdb2b3e8a9d6f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:13:53.970508 13553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d22f81e4955484bbd5f8b96043e71dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:53.970556 13755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d22f81e4955484bbd5f8b96043e71dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:53.970556 13982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d22f81e4955484bbd5f8b96043e71dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:53.978389 14118 tablet_bootstrap.cc:492] T 0d22f81e4955484bbd5f8b96043e71dd P 373718738acd43abb245a898c90fa249: Bootstrap starting.
I20260812 11:13:53.979029 14117 tablet_bootstrap.cc:492] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511: Bootstrap starting.
I20260812 11:13:53.979668 14121 tablet_bootstrap.cc:492] T 0d22f81e4955484bbd5f8b96043e71dd P c5db80c089ef445686ab9b90db2d144c: Bootstrap starting.
I20260812 11:13:53.981714 14121 tablet_bootstrap.cc:654] T 0d22f81e4955484bbd5f8b96043e71dd P c5db80c089ef445686ab9b90db2d144c: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:53.981822 14117 tablet_bootstrap.cc:654] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:53.982615 14121 log.cc:824] T 0d22f81e4955484bbd5f8b96043e71dd P c5db80c089ef445686ab9b90db2d144c: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:53.983443 14117 log.cc:824] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:53.985155 14121 tablet_bootstrap.cc:492] T 0d22f81e4955484bbd5f8b96043e71dd P c5db80c089ef445686ab9b90db2d144c: No bootstrap required, opened a new log
I20260812 11:13:53.985147 14118 tablet_bootstrap.cc:654] T 0d22f81e4955484bbd5f8b96043e71dd P 373718738acd43abb245a898c90fa249: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:53.985314 14117 tablet_bootstrap.cc:492] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511: No bootstrap required, opened a new log
I20260812 11:13:53.985395 14121 ts_tablet_manager.cc:1397] T 0d22f81e4955484bbd5f8b96043e71dd P c5db80c089ef445686ab9b90db2d144c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 11:13:53.985532 14117 ts_tablet_manager.cc:1397] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 11:13:53.986307 14118 log.cc:824] T 0d22f81e4955484bbd5f8b96043e71dd P 373718738acd43abb245a898c90fa249: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:53.988373 14118 tablet_bootstrap.cc:492] T 0d22f81e4955484bbd5f8b96043e71dd P 373718738acd43abb245a898c90fa249: No bootstrap required, opened a new log
I20260812 11:13:53.988585 14118 ts_tablet_manager.cc:1397] T 0d22f81e4955484bbd5f8b96043e71dd P 373718738acd43abb245a898c90fa249: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260812 11:13:53.988909 14117 raft_consensus.cc:348] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5ca16bb6a1466d8cdd7c3c4ee4f511" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 45233 } } peers { permanent_uuid: "373718738acd43abb245a898c90fa249" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45133 } } peers { permanent_uuid: "c5db80c089ef445686ab9b90db2d144c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 34177 } }
I20260812 11:13:53.989148 14117 raft_consensus.cc:374] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:53.989202 14117 raft_consensus.cc:729] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd5ca16bb6a1466d8cdd7c3c4ee4f511, State: Initialized, Role: FOLLOWER
I20260812 11:13:53.989665 14117 consensus_queue.cc:260] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5ca16bb6a1466d8cdd7c3c4ee4f511" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 45233 } } peers { permanent_uuid: "373718738acd43abb245a898c90fa249" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45133 } } peers { permanent_uuid: "c5db80c089ef445686ab9b90db2d144c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 34177 } }
I20260812 11:13:53.990335 13863 heartbeater.cc:503] Master 127.17.44.254:41899 was elected leader, sending a full tablet report...
I20260812 11:13:53.990289 14121 raft_consensus.cc:348] T 0d22f81e4955484bbd5f8b96043e71dd P c5db80c089ef445686ab9b90db2d144c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5ca16bb6a1466d8cdd7c3c4ee4f511" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 45233 } } peers { permanent_uuid: "373718738acd43abb245a898c90fa249" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45133 } } peers { permanent_uuid: "c5db80c089ef445686ab9b90db2d144c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 34177 } }
I20260812 11:13:53.990561 14121 raft_consensus.cc:374] T 0d22f81e4955484bbd5f8b96043e71dd P c5db80c089ef445686ab9b90db2d144c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:53.990623 14121 raft_consensus.cc:729] T 0d22f81e4955484bbd5f8b96043e71dd P c5db80c089ef445686ab9b90db2d144c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5db80c089ef445686ab9b90db2d144c, State: Initialized, Role: FOLLOWER
I20260812 11:13:53.990697 14117 ts_tablet_manager.cc:1428] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 11:13:53.991098 14121 consensus_queue.cc:260] T 0d22f81e4955484bbd5f8b96043e71dd P c5db80c089ef445686ab9b90db2d144c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5ca16bb6a1466d8cdd7c3c4ee4f511" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 45233 } } peers { permanent_uuid: "373718738acd43abb245a898c90fa249" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45133 } } peers { permanent_uuid: "c5db80c089ef445686ab9b90db2d144c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 34177 } }
I20260812 11:13:53.992067 13653 heartbeater.cc:503] Master 127.17.44.254:41899 was elected leader, sending a full tablet report...
I20260812 11:13:53.992193 14121 ts_tablet_manager.cc:1428] T 0d22f81e4955484bbd5f8b96043e71dd P c5db80c089ef445686ab9b90db2d144c: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 11:13:53.993844 14118 raft_consensus.cc:348] T 0d22f81e4955484bbd5f8b96043e71dd P 373718738acd43abb245a898c90fa249 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5ca16bb6a1466d8cdd7c3c4ee4f511" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 45233 } } peers { permanent_uuid: "373718738acd43abb245a898c90fa249" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45133 } } peers { permanent_uuid: "c5db80c089ef445686ab9b90db2d144c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 34177 } }
I20260812 11:13:53.994092 14118 raft_consensus.cc:374] T 0d22f81e4955484bbd5f8b96043e71dd P 373718738acd43abb245a898c90fa249 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:53.994158 14118 raft_consensus.cc:729] T 0d22f81e4955484bbd5f8b96043e71dd P 373718738acd43abb245a898c90fa249 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373718738acd43abb245a898c90fa249, State: Initialized, Role: FOLLOWER
I20260812 11:13:53.994645 14118 consensus_queue.cc:260] T 0d22f81e4955484bbd5f8b96043e71dd P 373718738acd43abb245a898c90fa249 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5ca16bb6a1466d8cdd7c3c4ee4f511" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 45233 } } peers { permanent_uuid: "373718738acd43abb245a898c90fa249" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45133 } } peers { permanent_uuid: "c5db80c089ef445686ab9b90db2d144c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 34177 } }
I20260812 11:13:53.995335 14087 heartbeater.cc:503] Master 127.17.44.254:41899 was elected leader, sending a full tablet report...
I20260812 11:13:53.996577 14118 ts_tablet_manager.cc:1428] T 0d22f81e4955484bbd5f8b96043e71dd P 373718738acd43abb245a898c90fa249: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20260812 11:13:54.000445 14130 raft_consensus.cc:482] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:13:54.000571 14130 raft_consensus.cc:504] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5ca16bb6a1466d8cdd7c3c4ee4f511" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 45233 } } peers { permanent_uuid: "373718738acd43abb245a898c90fa249" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45133 } } peers { permanent_uuid: "c5db80c089ef445686ab9b90db2d144c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 34177 } }
I20260812 11:13:54.001609 14130 leader_election.cc:288] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 373718738acd43abb245a898c90fa249 (127.17.44.195:45133), c5db80c089ef445686ab9b90db2d144c (127.17.44.193:34177)
I20260812 11:13:54.005609 13865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:54.001861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:34177 (local address 127.17.44.194:56527)
0812 11:13:54.002020 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:54.002031 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:54.002131 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:54.002394 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:54.002397 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:54.002413 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:54.002533 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:54.002538 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:54.003486 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:54.003490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:54.004382 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:54.004391 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:54.005153 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:54.005156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:54.005265 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:54.005293 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 11:13:54.005387 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 11:13:54.005475 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260812 11:13:54.005610 14114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:54.002023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:56527 (local address 127.17.44.193:34177)
0812 11:13:54.002162 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:54.002169 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:54.002185 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:13:54.002232 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:54.002235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:54.002275 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:54.002364 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:54.002691 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:54.003382 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:54.004509 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:54.005037 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:54.005128 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:54.005168 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:54.005396 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 11:13:54.005489 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 11:13:54.006016 14115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:54.002165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:45871 (local address 127.17.44.195:45133)
0812 11:13:54.002355 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:54.002359 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:54.002580 (+ 221us) server_negotiation.cc:407] Connection header received
0812 11:13:54.002695 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:54.002698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:54.002743 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:54.002800 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:54.003119 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:54.003676 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:54.004846 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:54.005373 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:54.005484 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:54.005530 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:54.005806 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 11:13:54.005910 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 11:13:54.006359 14134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:54.001903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:45133 (local address 127.17.44.194:45871)
0812 11:13:54.002485 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:54.002496 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:54.002585 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:54.002837 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:54.002839 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:54.002858 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:54.003008 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:54.003015 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:54.003840 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:54.003842 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:54.004705 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:54.004716 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:54.005490 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:54.005493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:54.005591 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:54.005614 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 11:13:54.006174 (+ 560us) client_negotiation.cc:240] Negotiation successful
0812 11:13:54.006253 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":120,"threads_started":1}
I20260812 11:13:54.006302 13579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d22f81e4955484bbd5f8b96043e71dd" candidate_uuid: "bd5ca16bb6a1466d8cdd7c3c4ee4f511" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5db80c089ef445686ab9b90db2d144c" is_pre_election: true
I20260812 11:13:54.006603 13579 raft_consensus.cc:2454] T 0d22f81e4955484bbd5f8b96043e71dd P c5db80c089ef445686ab9b90db2d144c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd5ca16bb6a1466d8cdd7c3c4ee4f511 in term 0.
I20260812 11:13:54.006911 14012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d22f81e4955484bbd5f8b96043e71dd" candidate_uuid: "bd5ca16bb6a1466d8cdd7c3c4ee4f511" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373718738acd43abb245a898c90fa249" is_pre_election: true
I20260812 11:13:54.007119 13690 leader_election.cc:302] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd5ca16bb6a1466d8cdd7c3c4ee4f511, c5db80c089ef445686ab9b90db2d144c; no voters:
I20260812 11:13:54.007210 14012 raft_consensus.cc:2454] T 0d22f81e4955484bbd5f8b96043e71dd P 373718738acd43abb245a898c90fa249 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd5ca16bb6a1466d8cdd7c3c4ee4f511 in term 0.
I20260812 11:13:54.007390 14130 raft_consensus.cc:2781] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:13:54.007500 14130 raft_consensus.cc:482] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:13:54.007560 14130 raft_consensus.cc:3037] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:54.008677 14130 raft_consensus.cc:504] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5ca16bb6a1466d8cdd7c3c4ee4f511" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 45233 } } peers { permanent_uuid: "373718738acd43abb245a898c90fa249" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45133 } } peers { permanent_uuid: "c5db80c089ef445686ab9b90db2d144c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 34177 } }
I20260812 11:13:54.009099 14130 leader_election.cc:288] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511 [CANDIDATE]: Term 1 election: Requested vote from peers 373718738acd43abb245a898c90fa249 (127.17.44.195:45133), c5db80c089ef445686ab9b90db2d144c (127.17.44.193:34177)
I20260812 11:13:54.009533 13579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d22f81e4955484bbd5f8b96043e71dd" candidate_uuid: "bd5ca16bb6a1466d8cdd7c3c4ee4f511" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5db80c089ef445686ab9b90db2d144c"
I20260812 11:13:54.009580 14012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d22f81e4955484bbd5f8b96043e71dd" candidate_uuid: "bd5ca16bb6a1466d8cdd7c3c4ee4f511" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373718738acd43abb245a898c90fa249"
I20260812 11:13:54.009706 14012 raft_consensus.cc:3037] T 0d22f81e4955484bbd5f8b96043e71dd P 373718738acd43abb245a898c90fa249 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:54.009685 13579 raft_consensus.cc:3037] T 0d22f81e4955484bbd5f8b96043e71dd P c5db80c089ef445686ab9b90db2d144c [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:54.011168 14012 raft_consensus.cc:2454] T 0d22f81e4955484bbd5f8b96043e71dd P 373718738acd43abb245a898c90fa249 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd5ca16bb6a1466d8cdd7c3c4ee4f511 in term 1.
I20260812 11:13:54.011219 13579 raft_consensus.cc:2454] T 0d22f81e4955484bbd5f8b96043e71dd P c5db80c089ef445686ab9b90db2d144c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd5ca16bb6a1466d8cdd7c3c4ee4f511 in term 1.
I20260812 11:13:54.011518 13690 leader_election.cc:302] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd5ca16bb6a1466d8cdd7c3c4ee4f511, c5db80c089ef445686ab9b90db2d144c; no voters:
I20260812 11:13:54.012286 14130 raft_consensus.cc:2781] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:13:54.012709 14130 raft_consensus.cc:686] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511 [term 1 LEADER]: Becoming Leader. State: Replica: bd5ca16bb6a1466d8cdd7c3c4ee4f511, State: Running, Role: LEADER
I20260812 11:13:54.013289 14130 consensus_queue.cc:237] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5ca16bb6a1466d8cdd7c3c4ee4f511" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 45233 } } peers { permanent_uuid: "373718738acd43abb245a898c90fa249" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45133 } } peers { permanent_uuid: "c5db80c089ef445686ab9b90db2d144c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 34177 } }
I20260812 11:13:54.016536 13361 catalog_manager.cc:5495] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd5ca16bb6a1466d8cdd7c3c4ee4f511 (127.17.44.194). New cstate: current_term: 1 leader_uuid: "bd5ca16bb6a1466d8cdd7c3c4ee4f511" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5ca16bb6a1466d8cdd7c3c4ee4f511" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373718738acd43abb245a898c90fa249" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5db80c089ef445686ab9b90db2d144c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 34177 } health_report { overall_health: UNKNOWN } } }
I20260812 11:13:54.031873 14111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:54.027992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60702 (local address 127.17.44.194:45233)
0812 11:13:54.028129 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:54.028134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:54.028148 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:13:54.028304 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:54.028306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:54.028359 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:54.028474 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:13:54.028737 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:54.029294 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:54.030203 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:54.030523 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:54.030665 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:54.031002 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:13:54.031418 (+ 416us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:13:54.031582 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:54.031684 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 11:13:54.031722 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 11:13:54.038695 13579 raft_consensus.cc:1261] T 0d22f81e4955484bbd5f8b96043e71dd P c5db80c089ef445686ab9b90db2d144c [term 1 FOLLOWER]: Refusing update from remote peer bd5ca16bb6a1466d8cdd7c3c4ee4f511: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:13:54.038757 14012 raft_consensus.cc:1261] T 0d22f81e4955484bbd5f8b96043e71dd P 373718738acd43abb245a898c90fa249 [term 1 FOLLOWER]: Refusing update from remote peer bd5ca16bb6a1466d8cdd7c3c4ee4f511: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:13:54.039587 14136 consensus_queue.cc:1035] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5db80c089ef445686ab9b90db2d144c" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 34177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:13:54.039817 14130 consensus_queue.cc:1035] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511 [LEADER]: Connected to new peer: Peer: permanent_uuid: "373718738acd43abb245a898c90fa249" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:13:54.046837 14142 mvcc.cc:205] Tried to move back new op lower bound from 7317640126615216128 to 7317640126521376768. Current Snapshot: MvccSnapshot[applied={T|T < 7317640126615216128}]
I20260812 11:13:54.049293 14145 mvcc.cc:205] Tried to move back new op lower bound from 7317640126615216128 to 7317640126521376768. Current Snapshot: MvccSnapshot[applied={T|T < 7317640126615216128 or (T in {7317640126615216128})}]
I20260812 11:13:54.055637 13361 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:13:54.055835 13361 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:13:54.058492 13361 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4fdec254e4c34b10b9333bca3d5e8a45: Sending DeleteTablet for 3 replicas of tablet 0d22f81e4955484bbd5f8b96043e71dd
I20260812 11:13:54.059422 13982 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d22f81e4955484bbd5f8b96043e71dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:13:54 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:40600
I20260812 11:13:54.059428 13553 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d22f81e4955484bbd5f8b96043e71dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:13:54 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:43970
I20260812 11:13:54.059909 14154 tablet_replica.cc:330] T 0d22f81e4955484bbd5f8b96043e71dd P 373718738acd43abb245a898c90fa249: stopping tablet replica
I20260812 11:13:54.060211 13755 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d22f81e4955484bbd5f8b96043e71dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:13:54 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:60692
I20260812 11:13:54.060271 14154 raft_consensus.cc:2229] T 0d22f81e4955484bbd5f8b96043e71dd P 373718738acd43abb245a898c90fa249 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:13:54.060698 14154 raft_consensus.cc:2258] T 0d22f81e4955484bbd5f8b96043e71dd P 373718738acd43abb245a898c90fa249 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:13:54.061201 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 13447
I20260812 11:13:54.065464 14154 ts_tablet_manager.cc:1905] T 0d22f81e4955484bbd5f8b96043e71dd P 373718738acd43abb245a898c90fa249: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:13:54.069559 14154 ts_tablet_manager.cc:1918] T 0d22f81e4955484bbd5f8b96043e71dd P 373718738acd43abb245a898c90fa249: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:13:54.069774 14154 log.cc:1196] T 0d22f81e4955484bbd5f8b96043e71dd P 373718738acd43abb245a898c90fa249: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/0d22f81e4955484bbd5f8b96043e71dd
I20260812 11:13:54.070322 14154 ts_tablet_manager.cc:1939] T 0d22f81e4955484bbd5f8b96043e71dd P 373718738acd43abb245a898c90fa249: Deleting consensus metadata
I20260812 11:13:54.076916 13341 catalog_manager.cc:4841] TS 373718738acd43abb245a898c90fa249 (127.17.44.195:45133): tablet 0d22f81e4955484bbd5f8b96043e71dd (table test-table [id=15ff31efee9b4c04bccdb2b3e8a9d6f1]) successfully deleted
I20260812 11:13:54.078269 14157 tablet_replica.cc:330] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511: stopping tablet replica
I20260812 11:13:54.078644 14157 raft_consensus.cc:2229] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:13:54.079201 14157 raft_consensus.cc:2258] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:13:54.080463 14157 ts_tablet_manager.cc:1905] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:13:54.082713 14157 ts_tablet_manager.cc:1918] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:13:54.082836 14157 log.cc:1196] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/0d22f81e4955484bbd5f8b96043e71dd
I20260812 11:13:54.083169 14157 ts_tablet_manager.cc:1939] T 0d22f81e4955484bbd5f8b96043e71dd P bd5ca16bb6a1466d8cdd7c3c4ee4f511: Deleting consensus metadata
I20260812 11:13:54.084373 13346 catalog_manager.cc:4841] TS bd5ca16bb6a1466d8cdd7c3c4ee4f511 (127.17.44.194:45233): tablet 0d22f81e4955484bbd5f8b96043e71dd (table test-table [id=15ff31efee9b4c04bccdb2b3e8a9d6f1]) successfully deleted
W20260812 11:13:54.085743 13343 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.44.193:34177 (error 108)
I20260812 11:13:54.085983 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 13663
I20260812 11:13:54.095538 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:54.095223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:34177 (local address 127.0.0.1:43980)
0812 11:13:54.095373 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:54.095456 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.193:34177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260812 11:13:54.095713 13343 catalog_manager.cc:4584] TS c5db80c089ef445686ab9b90db2d144c (127.17.44.193:34177): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0d22f81e4955484bbd5f8b96043e71dd: Network error: Client connection negotiation failed: client connection to 127.17.44.193:34177: connect: Connection refused (error 111)
I20260812 11:13:54.102154 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 13871
I20260812 11:13:54.116622 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 13321
2026-08-12T11:13:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2692 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T11:13:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:13:54Z Disabled control of system clock
I20260812 11:13:54.157927 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:43731
--webserver_interface=127.17.44.254
--webserver_port=0
--builtin_ntp_servers=127.17.44.212:35113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:43731
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 11:13:54.280012 14164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:54.280269 14164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:54.280303 14164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:54.284801 14164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:13:54.284910 14164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:54.284929 14164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:54.284947 14164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:13:54.284965 14164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:13:54.290715 14164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:35113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:43731
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:43731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:54.292162 14164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:54.293849 14164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:54.303615 14176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:54.303823 14172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:54.303941 14164 server_base.cc:1034] running on GCE node
W20260812 11:13:54.303615 14174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:54.304811 14164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:54.306182 14164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:54.307420 14164 hybrid_clock.cc:648] HybridClock initialized: now 1786533234307368 us; error 61 us; skew 500 ppm
I20260812 11:13:54.309787 14164 webserver.cc:460] Webserver started at http://127.17.44.254:46761/ using document root <none> and password file <none>
I20260812 11:13:54.310448 14164 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:54.310561 14164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:54.310825 14164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:13:54.313119 14164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "5c06f39bf19a4ba7b9cc991fe436f821"
format_stamp: "Formatted at 2026-08-12 11:13:54 on dist-test-slave-drl0"
server_key: "2cc715f6a37b229d15d428ae8dd647e4"
server_key_iv: "890a2dceb6cbdae00f7f825540b45323"
server_key_version: "encryptionkey@0"
I20260812 11:13:54.313711 14164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "5c06f39bf19a4ba7b9cc991fe436f821"
format_stamp: "Formatted at 2026-08-12 11:13:54 on dist-test-slave-drl0"
server_key: "2cc715f6a37b229d15d428ae8dd647e4"
server_key_iv: "890a2dceb6cbdae00f7f825540b45323"
server_key_version: "encryptionkey@0"
I20260812 11:13:54.317572 14164 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 11:13:54.320379 14189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:54.321862 14164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 11:13:54.322023 14164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "5c06f39bf19a4ba7b9cc991fe436f821"
format_stamp: "Formatted at 2026-08-12 11:13:54 on dist-test-slave-drl0"
server_key: "2cc715f6a37b229d15d428ae8dd647e4"
server_key_iv: "890a2dceb6cbdae00f7f825540b45323"
server_key_version: "encryptionkey@0"
I20260812 11:13:54.322142 14164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:54.340433 14164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:54.341173 14164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:54.341378 14164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:54.349200 14164 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:43731
I20260812 11:13:54.349228 14270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:43731 every 8 connection(s)
I20260812 11:13:54.350134 14164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:13:54.353269 14273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:54.354990 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 14164
I20260812 11:13:54.355145 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:13:54.355443 17587 external_mini_cluster.cc:1428] Setting key 06ed3fdc895108b73ffe0284a7fc6dce
I20260812 11:13:54.360059 14273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c06f39bf19a4ba7b9cc991fe436f821: Bootstrap starting.
I20260812 11:13:54.362640 14273 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c06f39bf19a4ba7b9cc991fe436f821: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:54.363564 14273 log.cc:824] T 00000000000000000000000000000000 P 5c06f39bf19a4ba7b9cc991fe436f821: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:54.365449 14278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:54.356892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50882 (local address 127.17.44.254:43731)
0812 11:13:54.357577 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:54.357594 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:54.357632 (+ 38us) server_negotiation.cc:407] Connection header received
0812 11:13:54.358339 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:54.358357 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:54.358778 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:54.359162 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:54.359524 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:54.360602 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:54.361846 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:54.362307 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:54.362447 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:54.363571 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:54.363581 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:54.363593 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:13:54.364126 (+ 533us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:54.364182 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:13:54.364204 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:54.364361 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:54.364512 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 11:13:54.364766 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":151,"threads_started":1}
I20260812 11:13:54.365624 14273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c06f39bf19a4ba7b9cc991fe436f821: No bootstrap required, opened a new log
I20260812 11:13:54.369227 14273 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c06f39bf19a4ba7b9cc991fe436f821 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c06f39bf19a4ba7b9cc991fe436f821" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43731 } }
I20260812 11:13:54.369446 14273 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c06f39bf19a4ba7b9cc991fe436f821 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:54.369503 14273 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c06f39bf19a4ba7b9cc991fe436f821 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c06f39bf19a4ba7b9cc991fe436f821, State: Initialized, Role: FOLLOWER
I20260812 11:13:54.370131 14273 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c06f39bf19a4ba7b9cc991fe436f821 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c06f39bf19a4ba7b9cc991fe436f821" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43731 } }
I20260812 11:13:54.370321 14273 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c06f39bf19a4ba7b9cc991fe436f821 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:13:54.370397 14273 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c06f39bf19a4ba7b9cc991fe436f821 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:13:54.370496 14273 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c06f39bf19a4ba7b9cc991fe436f821 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:54.371450 14273 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c06f39bf19a4ba7b9cc991fe436f821 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c06f39bf19a4ba7b9cc991fe436f821" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43731 } }
I20260812 11:13:54.371734 14273 leader_election.cc:302] T 00000000000000000000000000000000 P 5c06f39bf19a4ba7b9cc991fe436f821 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c06f39bf19a4ba7b9cc991fe436f821; no voters:
I20260812 11:13:54.371977 14273 leader_election.cc:288] T 00000000000000000000000000000000 P 5c06f39bf19a4ba7b9cc991fe436f821 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:13:54.372135 14280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c06f39bf19a4ba7b9cc991fe436f821 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:13:54.372411 14280 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c06f39bf19a4ba7b9cc991fe436f821 [term 1 LEADER]: Becoming Leader. State: Replica: 5c06f39bf19a4ba7b9cc991fe436f821, State: Running, Role: LEADER
I20260812 11:13:54.372720 14280 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c06f39bf19a4ba7b9cc991fe436f821 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c06f39bf19a4ba7b9cc991fe436f821" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43731 } }
I20260812 11:13:54.373322 14273 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c06f39bf19a4ba7b9cc991fe436f821 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:13:54.374416 14282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c06f39bf19a4ba7b9cc991fe436f821 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c06f39bf19a4ba7b9cc991fe436f821" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c06f39bf19a4ba7b9cc991fe436f821" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43731 } } }
I20260812 11:13:54.374542 14282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c06f39bf19a4ba7b9cc991fe436f821 [sys.catalog]: This master's current role is: LEADER
I20260812 11:13:54.374980 14293 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:13:54.374970 14283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c06f39bf19a4ba7b9cc991fe436f821 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c06f39bf19a4ba7b9cc991fe436f821. Latest consensus state: current_term: 1 leader_uuid: "5c06f39bf19a4ba7b9cc991fe436f821" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c06f39bf19a4ba7b9cc991fe436f821" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 43731 } } }
I20260812 11:13:54.375198 14283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c06f39bf19a4ba7b9cc991fe436f821 [sys.catalog]: This master's current role is: LEADER
I20260812 11:13:54.378051 14293 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:13:54.383782 14293 catalog_manager.cc:1347] Generated new cluster ID: 09cebda26f134469828d231ca43b602d
I20260812 11:13:54.383854 14293 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:13:54.390638 14293 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:13:54.391595 14293 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:13:54.401592 14293 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c06f39bf19a4ba7b9cc991fe436f821: Generated new TSK 0
I20260812 11:13:54.402256 14293 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:13:54.407228 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:43731
--builtin_ntp_servers=127.17.44.212:35113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 11:13:54.533027 14307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:54.533294 14307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:54.533334 14307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:54.538350 14307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:54.538415 14307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:54.538472 14307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:13:54.544843 14307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:35113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:43731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:54.546110 14307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:54.547662 14307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:54.557690 14322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:54.557690 14320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:54.557909 14307 server_base.cc:1034] running on GCE node
W20260812 11:13:54.557736 14317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:54.558394 14307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:54.559062 14307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:54.560261 14307 hybrid_clock.cc:648] HybridClock initialized: now 1786533234560241 us; error 31 us; skew 500 ppm
I20260812 11:13:54.562456 14307 webserver.cc:460] Webserver started at http://127.17.44.193:40051/ using document root <none> and password file <none>
I20260812 11:13:54.563120 14307 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:54.563213 14307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:54.563443 14307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:13:54.565601 14307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "6045ad5c346345be95d1a784bdc1b1f6"
format_stamp: "Formatted at 2026-08-12 11:13:54 on dist-test-slave-drl0"
server_key: "36d4f7e104ec0eec0e4981c2fad94b0c"
server_key_iv: "5a671150ff5d5eeab778609bbdacfaac"
server_key_version: "encryptionkey@0"
I20260812 11:13:54.566622 14307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "6045ad5c346345be95d1a784bdc1b1f6"
format_stamp: "Formatted at 2026-08-12 11:13:54 on dist-test-slave-drl0"
server_key: "36d4f7e104ec0eec0e4981c2fad94b0c"
server_key_iv: "5a671150ff5d5eeab778609bbdacfaac"
server_key_version: "encryptionkey@0"
I20260812 11:13:54.571029 14307 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260812 11:13:54.574069 14331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:54.575539 14307 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:13:54.575670 14307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "6045ad5c346345be95d1a784bdc1b1f6"
format_stamp: "Formatted at 2026-08-12 11:13:54 on dist-test-slave-drl0"
server_key: "36d4f7e104ec0eec0e4981c2fad94b0c"
server_key_iv: "5a671150ff5d5eeab778609bbdacfaac"
server_key_version: "encryptionkey@0"
I20260812 11:13:54.575780 14307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:54.587788 14307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:54.588522 14307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:54.588732 14307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:54.589311 14307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:13:54.590230 14307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:13:54.590305 14307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:54.590376 14307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:13:54.590427 14307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:54.600313 14307 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:34167
I20260812 11:13:54.600337 14511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:34167 every 8 connection(s)
I20260812 11:13:54.601214 14307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:13:54.605985 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 14307
I20260812 11:13:54.606163 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:13:54.606479 17587 external_mini_cluster.cc:1428] Setting key 1cfeddcb2ec624c62463abe8d0f36126
I20260812 11:13:54.609220 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:43731
--builtin_ntp_servers=127.17.44.212:35113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 11:13:54.612591 14278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:54.603297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:42055 (local address 127.17.44.254:43731)
0812 11:13:54.603452 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:54.603457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:54.605071 (+ 1614us) server_negotiation.cc:407] Connection header received
0812 11:13:54.606988 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:54.606993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:54.607261 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:54.607387 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:54.608629 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:54.609195 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:54.610495 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:54.610718 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:54.610827 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:54.612080 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:54.612088 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:54.612090 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:13:54.612102 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:54.612120 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:13:54.612127 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:54.612207 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:54.612376 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 11:13:54.612445 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 11:13:54.613469 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:54.603723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:43731 (local address 127.17.44.193:42055)
0812 11:13:54.604816 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:54.604882 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:54.606621 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:54.607693 (+ 1072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:54.607706 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:54.607825 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:13:54.608475 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:54.608491 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:54.609329 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:54.609333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:54.610344 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:54.610354 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:54.610881 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:54.610885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:54.611115 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:54.611793 (+ 678us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:13:54.611795 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:13:54.611837 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:13:54.611909 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:13:54.612211 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:13:54.612223 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:13:54.612231 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 11:13:54.612384 (+ 153us) client_negotiation.cc:240] Negotiation successful
0812 11:13:54.612674 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":128,"threads_started":1}
I20260812 11:13:54.614996 14512 heartbeater.cc:347] Connected to a master server at 127.17.44.254:43731
I20260812 11:13:54.615257 14512 heartbeater.cc:464] Registering TS with master...
I20260812 11:13:54.615805 14512 heartbeater.cc:511] Master 127.17.44.254:43731 requested a full tablet report, sending...
I20260812 11:13:54.617530 14219 ts_manager.cc:194] Registered new tserver with Master: 6045ad5c346345be95d1a784bdc1b1f6 (127.17.44.193:34167)
I20260812 11:13:54.619282 14219 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.44.193:42055
W20260812 11:13:54.733979 14520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:54.734232 14520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:54.734270 14520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:54.738274 14520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:54.738327 14520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:54.738381 14520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:13:54.743125 14520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:35113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:43731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:54.744269 14520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:54.745806 14520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:54.756567 14528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:54.756554 14527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:54.756932 14520 server_base.cc:1034] running on GCE node
W20260812 11:13:54.757081 14531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:54.757495 14520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:54.758111 14520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:54.759285 14520 hybrid_clock.cc:648] HybridClock initialized: now 1786533234759245 us; error 43 us; skew 500 ppm
I20260812 11:13:54.761153 14520 webserver.cc:460] Webserver started at http://127.17.44.194:39463/ using document root <none> and password file <none>
I20260812 11:13:54.761751 14520 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:54.761816 14520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:54.762044 14520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:13:54.764160 14520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "673f4b17487e467fa03b24fb69a0b921"
format_stamp: "Formatted at 2026-08-12 11:13:54 on dist-test-slave-drl0"
server_key: "b8e13e9977065c83d64d6f05b92ad43c"
server_key_iv: "f4a46ee08e8e959dccb95cdd065066d5"
server_key_version: "encryptionkey@0"
I20260812 11:13:54.764739 14520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "673f4b17487e467fa03b24fb69a0b921"
format_stamp: "Formatted at 2026-08-12 11:13:54 on dist-test-slave-drl0"
server_key: "b8e13e9977065c83d64d6f05b92ad43c"
server_key_iv: "f4a46ee08e8e959dccb95cdd065066d5"
server_key_version: "encryptionkey@0"
I20260812 11:13:54.768863 14520 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 11:13:54.771430 14543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:54.772845 14520 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:13:54.772996 14520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "673f4b17487e467fa03b24fb69a0b921"
format_stamp: "Formatted at 2026-08-12 11:13:54 on dist-test-slave-drl0"
server_key: "b8e13e9977065c83d64d6f05b92ad43c"
server_key_iv: "f4a46ee08e8e959dccb95cdd065066d5"
server_key_version: "encryptionkey@0"
I20260812 11:13:54.773110 14520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:54.785194 14520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:54.785836 14520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:54.786053 14520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:54.786626 14520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:13:54.787580 14520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:13:54.787654 14520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:54.787721 14520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:13:54.787768 14520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:54.797596 14520 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:34835
I20260812 11:13:54.797619 14720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:34835 every 8 connection(s)
I20260812 11:13:54.798475 14520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:13:54.798755 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 14520
I20260812 11:13:54.798869 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:13:54.799276 17587 external_mini_cluster.cc:1428] Setting key 92cb14b35d2c76a9fc67452f9300fe16
I20260812 11:13:54.801867 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:43731
--builtin_ntp_servers=127.17.44.212:35113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 11:13:54.811094 14278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:54.801209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:57163 (local address 127.17.44.254:43731)
0812 11:13:54.801352 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:54.801357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:54.802663 (+ 1306us) server_negotiation.cc:407] Connection header received
0812 11:13:54.804623 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:54.804628 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:54.804692 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:54.804807 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:54.805956 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:54.806610 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:54.808674 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:54.808928 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:54.809124 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:54.810474 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:54.810477 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:54.810484 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:13:54.810497 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:54.810516 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:13:54.810524 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:54.810589 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:54.810837 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 11:13:54.810899 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:13:54.812294 14727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:54.801681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:43731 (local address 127.17.44.194:57163)
0812 11:13:54.802472 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:54.802531 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:54.804390 (+ 1859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:54.804968 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:54.804974 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:54.805052 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:13:54.805689 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:54.805711 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:54.806798 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:54.806804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:54.808493 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:54.808509 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:54.809170 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:54.809177 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:54.809418 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:54.810127 (+ 709us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:13:54.810131 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:13:54.810192 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:13:54.810286 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:13:54.810622 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:13:54.810635 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:13:54.810645 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 11:13:54.810809 (+ 164us) client_negotiation.cc:240] Negotiation successful
0812 11:13:54.811163 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":123,"threads_started":1}
I20260812 11:13:54.813915 14722 heartbeater.cc:347] Connected to a master server at 127.17.44.254:43731
I20260812 11:13:54.814195 14722 heartbeater.cc:464] Registering TS with master...
I20260812 11:13:54.814760 14722 heartbeater.cc:511] Master 127.17.44.254:43731 requested a full tablet report, sending...
I20260812 11:13:54.815966 14219 ts_manager.cc:194] Registered new tserver with Master: 673f4b17487e467fa03b24fb69a0b921 (127.17.44.194:34835)
I20260812 11:13:54.816558 14219 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.44.194:57163
W20260812 11:13:54.928639 14729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:54.929631 14729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:54.929737 14729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:54.935694 14729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:54.935796 14729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:54.935897 14729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:13:54.943518 14729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:35113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:43731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:54.945150 14729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:54.947052 14729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:54.958990 14739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:54.958990 14736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:54.958993 14737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:54.959198 14729 server_base.cc:1034] running on GCE node
I20260812 11:13:54.959856 14729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:54.960479 14729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:54.961669 14729 hybrid_clock.cc:648] HybridClock initialized: now 1786533234961640 us; error 47 us; skew 500 ppm
I20260812 11:13:54.963689 14729 webserver.cc:460] Webserver started at http://127.17.44.195:40687/ using document root <none> and password file <none>
I20260812 11:13:54.964283 14729 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:54.964354 14729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:54.964543 14729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:13:54.966643 14729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "98806d25cf6b4c29b6b49ae7f8f89d5c"
format_stamp: "Formatted at 2026-08-12 11:13:54 on dist-test-slave-drl0"
server_key: "5aec64388b1fecfe7451099058ec0d12"
server_key_iv: "2dbceb435eaeaf8a98e66e9530a039a3"
server_key_version: "encryptionkey@0"
I20260812 11:13:54.967275 14729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "98806d25cf6b4c29b6b49ae7f8f89d5c"
format_stamp: "Formatted at 2026-08-12 11:13:54 on dist-test-slave-drl0"
server_key: "5aec64388b1fecfe7451099058ec0d12"
server_key_iv: "2dbceb435eaeaf8a98e66e9530a039a3"
server_key_version: "encryptionkey@0"
I20260812 11:13:54.972014 14729 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260812 11:13:54.974648 14752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:54.976090 14729 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 11:13:54.976267 14729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "98806d25cf6b4c29b6b49ae7f8f89d5c"
format_stamp: "Formatted at 2026-08-12 11:13:54 on dist-test-slave-drl0"
server_key: "5aec64388b1fecfe7451099058ec0d12"
server_key_iv: "2dbceb435eaeaf8a98e66e9530a039a3"
server_key_version: "encryptionkey@0"
I20260812 11:13:54.976372 14729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:54.992745 14729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:54.993412 14729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:54.993563 14729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:54.994228 14729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:13:54.995229 14729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:13:54.995283 14729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:54.995324 14729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:13:54.995339 14729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:55.005174 14729 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:45467
I20260812 11:13:55.005236 14922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:45467 every 8 connection(s)
I20260812 11:13:55.006160 14729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:13:55.010910 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 14729
I20260812 11:13:55.011106 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:13:55.011402 17587 external_mini_cluster.cc:1428] Setting key 70c64e12a135c6d45e7b23ba72c62738
I20260812 11:13:55.017745 14278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:55.008102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:37809 (local address 127.17.44.254:43731)
0812 11:13:55.008250 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:55.008253 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:55.009211 (+ 958us) server_negotiation.cc:407] Connection header received
0812 11:13:55.010698 (+ 1487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:55.010701 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:55.010763 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:55.010871 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:55.012003 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.012829 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.015005 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.015363 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.015511 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:55.017222 (+ 1711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:55.017226 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:55.017228 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:13:55.017243 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:55.017267 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:13:55.017278 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:55.017370 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:55.017583 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 11:13:55.017636 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 11:13:55.018671 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:55.008399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:43731 (local address 127.17.44.195:37809)
0812 11:13:55.009065 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:55.009106 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:55.010467 (+ 1361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:55.011107 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:55.011122 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:55.011244 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:13:55.011844 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:55.011866 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.013097 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.013112 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:55.014777 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:55.014823 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.015720 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.015725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:55.015986 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:55.016826 (+ 840us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:13:55.016830 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:13:55.016890 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:13:55.016976 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:13:55.017386 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:13:55.017397 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:13:55.017405 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 11:13:55.017585 (+ 180us) client_negotiation.cc:240] Negotiation successful
0812 11:13:55.017840 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":126,"threads_started":1}
I20260812 11:13:55.019953 14925 heartbeater.cc:347] Connected to a master server at 127.17.44.254:43731
I20260812 11:13:55.020165 14925 heartbeater.cc:464] Registering TS with master...
I20260812 11:13:55.020632 14925 heartbeater.cc:511] Master 127.17.44.254:43731 requested a full tablet report, sending...
I20260812 11:13:55.021652 14219 ts_manager.cc:194] Registered new tserver with Master: 98806d25cf6b4c29b6b49ae7f8f89d5c (127.17.44.195:45467)
I20260812 11:13:55.022238 14219 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.44.195:37809
I20260812 11:13:55.026508 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:13:55.033345 14278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:55.028859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50890 (local address 127.17.44.254:43731)
0812 11:13:55.028998 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:55.029002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:55.029225 (+ 223us) server_negotiation.cc:407] Connection header received
0812 11:13:55.029267 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:55.029270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:55.029308 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:55.029390 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:55.029720 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.030379 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.031702 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.031914 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.031998 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:55.032871 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:55.032874 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:55.032877 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:13:55.032888 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:55.032904 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:13:55.032912 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:55.033005 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:55.033204 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 11:13:55.033242 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 11:13:55.036469 14219 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50890:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:13:55.039503 14219 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:13:55.057001 14948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:55.050108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46334 (local address 127.17.44.194:34835)
0812 11:13:55.050740 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:55.050755 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:55.051231 (+ 476us) server_negotiation.cc:407] Connection header received
0812 11:13:55.051335 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:55.051341 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:55.051610 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:55.051789 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:55.052233 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.053389 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.055257 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.056105 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.056240 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:55.056474 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:55.056692 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 11:13:55.056820 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":192,"threads_started":1}
I20260812 11:13:55.057039 14946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:55.049800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:34835 (local address 127.0.0.1:46334)
0812 11:13:55.050992 (+ 1192us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:55.051009 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:55.051121 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:55.051903 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:55.051910 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:55.051938 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:55.052090 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:55.052095 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.053546 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.053552 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:55.055090 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:55.055099 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.056252 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.056254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:55.056415 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:55.056522 (+ 107us) client_negotiation.cc:769] Sending connection context
0812 11:13:55.056647 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 11:13:55.056850 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1075,"mutex_wait_us":143,"spinlock_wait_cycles":11136,"thread_start_us":273,"threads_started":1}
I20260812 11:13:55.057037 14945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:55.049795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:45467 (local address 127.0.0.1:48646)
0812 11:13:55.050449 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:55.050513 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:55.050682 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:55.052299 (+ 1617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:55.052303 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:55.052333 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:55.052459 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:55.052468 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.054110 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.054113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:55.055358 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:55.055365 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.056343 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.056347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:55.056418 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:55.056522 (+ 104us) client_negotiation.cc:769] Sending connection context
0812 11:13:55.056647 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 11:13:55.056850 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":81,"spinlock_wait_cycles":896,"thread_start_us":111,"threads_started":1}
I20260812 11:13:55.057647 14953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:55.050824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48646 (local address 127.17.44.195:45467)
0812 11:13:55.051747 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:55.051761 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:55.051790 (+ 29us) server_negotiation.cc:407] Connection header received
0812 11:13:55.051897 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:55.051904 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:55.052122 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:55.052617 (+ 495us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:55.052880 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.053989 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.055488 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.056218 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.056332 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:55.056472 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:55.057240 (+ 768us) server_negotiation.cc:299] Negotiation successful
0812 11:13:55.057490 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":790,"thread_start_us":177,"threads_started":1}
I20260812 11:13:55.059319 14950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:55.050838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:34167 (local address 127.0.0.1:37236)
0812 11:13:55.051344 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:55.051361 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:55.051472 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:55.052260 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:55.052263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:55.052284 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:55.052379 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:55.052385 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.053964 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.053968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:55.055337 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:55.055349 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.058795 (+ 3446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.058799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:55.058985 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:55.059012 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 11:13:55.059110 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 11:13:55.059183 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":58,"thread_start_us":160,"threads_started":1}
I20260812 11:13:55.059394 14955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:55.051392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37236 (local address 127.17.44.193:34167)
0812 11:13:55.051846 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:55.051856 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:55.051875 (+ 19us) server_negotiation.cc:407] Connection header received
0812 11:13:55.051940 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:55.051944 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:55.052122 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:55.052290 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:55.052486 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.053808 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.056541 (+ 2733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.058661 (+ 2120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.058785 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:55.058976 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:55.059150 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 11:13:55.059242 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":163,"threads_started":1}
I20260812 11:13:55.060441 14620 tablet_service.cc:1467] Processing CreateTablet for tablet fa290e1965704676b877617839b5db60 (DEFAULT_TABLE table=test-table [id=2e5030c118574207920d8a851161d3f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:13:55.060442 14826 tablet_service.cc:1467] Processing CreateTablet for tablet fa290e1965704676b877617839b5db60 (DEFAULT_TABLE table=test-table [id=2e5030c118574207920d8a851161d3f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:13:55.061578 14826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa290e1965704676b877617839b5db60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:55.061580 14620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa290e1965704676b877617839b5db60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:55.062196 14411 tablet_service.cc:1467] Processing CreateTablet for tablet fa290e1965704676b877617839b5db60 (DEFAULT_TABLE table=test-table [id=2e5030c118574207920d8a851161d3f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:13:55.063199 14411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa290e1965704676b877617839b5db60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:55.068207 14956 tablet_bootstrap.cc:492] T fa290e1965704676b877617839b5db60 P 673f4b17487e467fa03b24fb69a0b921: Bootstrap starting.
I20260812 11:13:55.069729 14961 tablet_bootstrap.cc:492] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c: Bootstrap starting.
I20260812 11:13:55.070408 14962 tablet_bootstrap.cc:492] T fa290e1965704676b877617839b5db60 P 6045ad5c346345be95d1a784bdc1b1f6: Bootstrap starting.
I20260812 11:13:55.070989 14956 tablet_bootstrap.cc:654] T fa290e1965704676b877617839b5db60 P 673f4b17487e467fa03b24fb69a0b921: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:55.071727 14961 tablet_bootstrap.cc:654] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:55.072196 14956 log.cc:824] T fa290e1965704676b877617839b5db60 P 673f4b17487e467fa03b24fb69a0b921: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:55.072530 14961 log.cc:824] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:55.072844 14962 tablet_bootstrap.cc:654] T fa290e1965704676b877617839b5db60 P 6045ad5c346345be95d1a784bdc1b1f6: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:55.073710 14962 log.cc:824] T fa290e1965704676b877617839b5db60 P 6045ad5c346345be95d1a784bdc1b1f6: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:55.074615 14956 tablet_bootstrap.cc:492] T fa290e1965704676b877617839b5db60 P 673f4b17487e467fa03b24fb69a0b921: No bootstrap required, opened a new log
I20260812 11:13:55.074671 14961 tablet_bootstrap.cc:492] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c: No bootstrap required, opened a new log
I20260812 11:13:55.074821 14956 ts_tablet_manager.cc:1397] T fa290e1965704676b877617839b5db60 P 673f4b17487e467fa03b24fb69a0b921: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260812 11:13:55.074820 14961 ts_tablet_manager.cc:1397] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 11:13:55.075671 14962 tablet_bootstrap.cc:492] T fa290e1965704676b877617839b5db60 P 6045ad5c346345be95d1a784bdc1b1f6: No bootstrap required, opened a new log
I20260812 11:13:55.075922 14962 ts_tablet_manager.cc:1397] T fa290e1965704676b877617839b5db60 P 6045ad5c346345be95d1a784bdc1b1f6: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:13:55.079425 14956 raft_consensus.cc:348] T fa290e1965704676b877617839b5db60 P 673f4b17487e467fa03b24fb69a0b921 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98806d25cf6b4c29b6b49ae7f8f89d5c" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45467 } } peers { permanent_uuid: "673f4b17487e467fa03b24fb69a0b921" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34835 } } peers { permanent_uuid: "6045ad5c346345be95d1a784bdc1b1f6" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 34167 } }
I20260812 11:13:55.079425 14961 raft_consensus.cc:348] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98806d25cf6b4c29b6b49ae7f8f89d5c" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45467 } } peers { permanent_uuid: "673f4b17487e467fa03b24fb69a0b921" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34835 } } peers { permanent_uuid: "6045ad5c346345be95d1a784bdc1b1f6" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 34167 } }
I20260812 11:13:55.079425 14962 raft_consensus.cc:348] T fa290e1965704676b877617839b5db60 P 6045ad5c346345be95d1a784bdc1b1f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98806d25cf6b4c29b6b49ae7f8f89d5c" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45467 } } peers { permanent_uuid: "673f4b17487e467fa03b24fb69a0b921" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34835 } } peers { permanent_uuid: "6045ad5c346345be95d1a784bdc1b1f6" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 34167 } }
I20260812 11:13:55.079705 14961 raft_consensus.cc:374] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:55.079705 14962 raft_consensus.cc:374] T fa290e1965704676b877617839b5db60 P 6045ad5c346345be95d1a784bdc1b1f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:55.079746 14956 raft_consensus.cc:374] T fa290e1965704676b877617839b5db60 P 673f4b17487e467fa03b24fb69a0b921 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:55.079767 14962 raft_consensus.cc:729] T fa290e1965704676b877617839b5db60 P 6045ad5c346345be95d1a784bdc1b1f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6045ad5c346345be95d1a784bdc1b1f6, State: Initialized, Role: FOLLOWER
I20260812 11:13:55.079799 14961 raft_consensus.cc:729] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98806d25cf6b4c29b6b49ae7f8f89d5c, State: Initialized, Role: FOLLOWER
I20260812 11:13:55.079819 14956 raft_consensus.cc:729] T fa290e1965704676b877617839b5db60 P 673f4b17487e467fa03b24fb69a0b921 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 673f4b17487e467fa03b24fb69a0b921, State: Initialized, Role: FOLLOWER
I20260812 11:13:55.080286 14962 consensus_queue.cc:260] T fa290e1965704676b877617839b5db60 P 6045ad5c346345be95d1a784bdc1b1f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98806d25cf6b4c29b6b49ae7f8f89d5c" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45467 } } peers { permanent_uuid: "673f4b17487e467fa03b24fb69a0b921" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34835 } } peers { permanent_uuid: "6045ad5c346345be95d1a784bdc1b1f6" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 34167 } }
I20260812 11:13:55.080312 14961 consensus_queue.cc:260] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98806d25cf6b4c29b6b49ae7f8f89d5c" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45467 } } peers { permanent_uuid: "673f4b17487e467fa03b24fb69a0b921" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34835 } } peers { permanent_uuid: "6045ad5c346345be95d1a784bdc1b1f6" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 34167 } }
I20260812 11:13:55.080330 14956 consensus_queue.cc:260] T fa290e1965704676b877617839b5db60 P 673f4b17487e467fa03b24fb69a0b921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98806d25cf6b4c29b6b49ae7f8f89d5c" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45467 } } peers { permanent_uuid: "673f4b17487e467fa03b24fb69a0b921" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34835 } } peers { permanent_uuid: "6045ad5c346345be95d1a784bdc1b1f6" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 34167 } }
I20260812 11:13:55.081073 14512 heartbeater.cc:503] Master 127.17.44.254:43731 was elected leader, sending a full tablet report...
I20260812 11:13:55.081436 14956 ts_tablet_manager.cc:1428] T fa290e1965704676b877617839b5db60 P 673f4b17487e467fa03b24fb69a0b921: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260812 11:13:55.081436 14961 ts_tablet_manager.cc:1428] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 11:13:55.081773 14925 heartbeater.cc:503] Master 127.17.44.254:43731 was elected leader, sending a full tablet report...
I20260812 11:13:55.081764 14722 heartbeater.cc:503] Master 127.17.44.254:43731 was elected leader, sending a full tablet report...
I20260812 11:13:55.082605 14962 ts_tablet_manager.cc:1428] T fa290e1965704676b877617839b5db60 P 6045ad5c346345be95d1a784bdc1b1f6: Time spent starting tablet: real 0.007s user 0.000s sys 0.005s
W20260812 11:13:55.102598 14513 tablet.cc:2369] T fa290e1965704676b877617839b5db60 P 6045ad5c346345be95d1a784bdc1b1f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:13:55.149857 14968 raft_consensus.cc:482] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:13:55.150219 14968 raft_consensus.cc:504] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98806d25cf6b4c29b6b49ae7f8f89d5c" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45467 } } peers { permanent_uuid: "673f4b17487e467fa03b24fb69a0b921" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34835 } } peers { permanent_uuid: "6045ad5c346345be95d1a784bdc1b1f6" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 34167 } }
I20260812 11:13:55.152011 14968 leader_election.cc:288] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 673f4b17487e467fa03b24fb69a0b921 (127.17.44.194:34835), 6045ad5c346345be95d1a784bdc1b1f6 (127.17.44.193:34167)
I20260812 11:13:55.156613 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:55.152209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:34835 (local address 127.17.44.195:46251)
0812 11:13:55.152387 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:55.152405 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:55.152513 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:55.152914 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:55.152917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:55.152937 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:55.153074 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:55.153093 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.154006 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.154009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:55.155153 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:55.155164 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.156157 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.156160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:55.156270 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:55.156299 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 11:13:55.156397 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 11:13:55.156472 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 11:13:55.156710 14948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:55.152405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:46251 (local address 127.17.44.194:34835)
0812 11:13:55.152589 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:55.152595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:55.152612 (+ 17us) server_negotiation.cc:407] Connection header received
0812 11:13:55.152685 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:55.152690 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:55.152751 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:55.152834 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:55.153219 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.153878 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.155322 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.156000 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.156134 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:55.156190 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:55.156472 (+ 282us) server_negotiation.cc:299] Negotiation successful
0812 11:13:55.156572 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 11:13:55.157372 14970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:55.152537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:34167 (local address 127.17.44.195:54689)
0812 11:13:55.152966 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:55.152981 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:55.153089 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:55.153340 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:55.153343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:55.153362 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:55.153478 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:55.153484 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.154520 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.154528 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:55.155809 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:55.155824 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.156937 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.156940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:55.157047 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:55.157073 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 11:13:55.157157 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 11:13:55.157234 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":164,"threads_started":1}
I20260812 11:13:55.157382 14955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:55.152682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:54689 (local address 127.17.44.193:34167)
0812 11:13:55.152826 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:55.152830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:55.153090 (+ 260us) server_negotiation.cc:407] Connection header received
0812 11:13:55.153176 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:55.153181 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:55.153241 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:55.153301 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:55.153639 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.154265 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.156014 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.156639 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.156772 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:55.156836 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:55.157194 (+ 358us) server_negotiation.cc:299] Negotiation successful
0812 11:13:55.157260 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 11:13:55.157382 14649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa290e1965704676b877617839b5db60" candidate_uuid: "98806d25cf6b4c29b6b49ae7f8f89d5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "673f4b17487e467fa03b24fb69a0b921" is_pre_election: true
I20260812 11:13:55.157843 14649 raft_consensus.cc:2454] T fa290e1965704676b877617839b5db60 P 673f4b17487e467fa03b24fb69a0b921 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98806d25cf6b4c29b6b49ae7f8f89d5c in term 0.
I20260812 11:13:55.158177 14442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa290e1965704676b877617839b5db60" candidate_uuid: "98806d25cf6b4c29b6b49ae7f8f89d5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6045ad5c346345be95d1a784bdc1b1f6" is_pre_election: true
I20260812 11:13:55.158445 14442 raft_consensus.cc:2454] T fa290e1965704676b877617839b5db60 P 6045ad5c346345be95d1a784bdc1b1f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98806d25cf6b4c29b6b49ae7f8f89d5c in term 0.
I20260812 11:13:55.158684 14759 leader_election.cc:302] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 673f4b17487e467fa03b24fb69a0b921, 98806d25cf6b4c29b6b49ae7f8f89d5c; no voters:
I20260812 11:13:55.159106 14968 raft_consensus.cc:2781] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:13:55.159227 14968 raft_consensus.cc:482] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:13:55.159263 14968 raft_consensus.cc:3037] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:55.160575 14968 raft_consensus.cc:504] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98806d25cf6b4c29b6b49ae7f8f89d5c" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45467 } } peers { permanent_uuid: "673f4b17487e467fa03b24fb69a0b921" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34835 } } peers { permanent_uuid: "6045ad5c346345be95d1a784bdc1b1f6" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 34167 } }
I20260812 11:13:55.161017 14968 leader_election.cc:288] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c [CANDIDATE]: Term 1 election: Requested vote from peers 673f4b17487e467fa03b24fb69a0b921 (127.17.44.194:34835), 6045ad5c346345be95d1a784bdc1b1f6 (127.17.44.193:34167)
I20260812 11:13:55.161520 14442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa290e1965704676b877617839b5db60" candidate_uuid: "98806d25cf6b4c29b6b49ae7f8f89d5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6045ad5c346345be95d1a784bdc1b1f6"
I20260812 11:13:55.161520 14649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa290e1965704676b877617839b5db60" candidate_uuid: "98806d25cf6b4c29b6b49ae7f8f89d5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "673f4b17487e467fa03b24fb69a0b921"
I20260812 11:13:55.161657 14442 raft_consensus.cc:3037] T fa290e1965704676b877617839b5db60 P 6045ad5c346345be95d1a784bdc1b1f6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:55.161692 14649 raft_consensus.cc:3037] T fa290e1965704676b877617839b5db60 P 673f4b17487e467fa03b24fb69a0b921 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:55.163376 14442 raft_consensus.cc:2454] T fa290e1965704676b877617839b5db60 P 6045ad5c346345be95d1a784bdc1b1f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98806d25cf6b4c29b6b49ae7f8f89d5c in term 1.
I20260812 11:13:55.163376 14649 raft_consensus.cc:2454] T fa290e1965704676b877617839b5db60 P 673f4b17487e467fa03b24fb69a0b921 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98806d25cf6b4c29b6b49ae7f8f89d5c in term 1.
I20260812 11:13:55.163691 14759 leader_election.cc:302] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 673f4b17487e467fa03b24fb69a0b921, 98806d25cf6b4c29b6b49ae7f8f89d5c; no voters:
I20260812 11:13:55.164039 14968 raft_consensus.cc:2781] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:13:55.164414 14968 raft_consensus.cc:686] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c [term 1 LEADER]: Becoming Leader. State: Replica: 98806d25cf6b4c29b6b49ae7f8f89d5c, State: Running, Role: LEADER
I20260812 11:13:55.164846 14968 consensus_queue.cc:237] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98806d25cf6b4c29b6b49ae7f8f89d5c" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45467 } } peers { permanent_uuid: "673f4b17487e467fa03b24fb69a0b921" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34835 } } peers { permanent_uuid: "6045ad5c346345be95d1a784bdc1b1f6" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 34167 } }
I20260812 11:13:55.168640 14218 catalog_manager.cc:5495] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c reported cstate change: term changed from 0 to 1, leader changed from <none> to 98806d25cf6b4c29b6b49ae7f8f89d5c (127.17.44.195). New cstate: current_term: 1 leader_uuid: "98806d25cf6b4c29b6b49ae7f8f89d5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98806d25cf6b4c29b6b49ae7f8f89d5c" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 45467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "673f4b17487e467fa03b24fb69a0b921" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6045ad5c346345be95d1a784bdc1b1f6" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 34167 } health_report { overall_health: UNKNOWN } } }
I20260812 11:13:55.183054 14953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:55.179211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48648 (local address 127.17.44.195:45467)
0812 11:13:55.179334 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:55.179338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:55.179392 (+ 54us) server_negotiation.cc:407] Connection header received
0812 11:13:55.179555 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:55.179557 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:55.179616 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:55.179800 (+ 184us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:13:55.179974 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.180547 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.181444 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:55.181655 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:55.181764 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:55.182081 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:13:55.182465 (+ 384us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:13:55.182621 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:55.182839 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 11:13:55.182894 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 11:13:55.190369 14649 raft_consensus.cc:1261] T fa290e1965704676b877617839b5db60 P 673f4b17487e467fa03b24fb69a0b921 [term 1 FOLLOWER]: Refusing update from remote peer 98806d25cf6b4c29b6b49ae7f8f89d5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:13:55.190369 14442 raft_consensus.cc:1261] T fa290e1965704676b877617839b5db60 P 6045ad5c346345be95d1a784bdc1b1f6 [term 1 FOLLOWER]: Refusing update from remote peer 98806d25cf6b4c29b6b49ae7f8f89d5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:13:55.191221 14968 consensus_queue.cc:1035] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6045ad5c346345be95d1a784bdc1b1f6" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 34167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:13:55.191412 14971 consensus_queue.cc:1035] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "673f4b17487e467fa03b24fb69a0b921" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:13:55.207211 14218 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:13:55.207404 14218 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:13:55.210757 14218 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c06f39bf19a4ba7b9cc991fe436f821: Sending DeleteTablet for 3 replicas of tablet fa290e1965704676b877617839b5db60
I20260812 11:13:55.211598 14826 tablet_service.cc:1514] Processing DeleteTablet for tablet fa290e1965704676b877617839b5db60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:13:55 UTC) from {username='slave'} at 127.0.0.1:48646
I20260812 11:13:55.211994 14411 tablet_service.cc:1514] Processing DeleteTablet for tablet fa290e1965704676b877617839b5db60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:13:55 UTC) from {username='slave'} at 127.0.0.1:37236
I20260812 11:13:55.212047 14620 tablet_service.cc:1514] Processing DeleteTablet for tablet fa290e1965704676b877617839b5db60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:13:55 UTC) from {username='slave'} at 127.0.0.1:46334
I20260812 11:13:55.212325 14989 tablet_replica.cc:330] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c: stopping tablet replica
I20260812 11:13:55.212680 14990 tablet_replica.cc:330] T fa290e1965704676b877617839b5db60 P 6045ad5c346345be95d1a784bdc1b1f6: stopping tablet replica
I20260812 11:13:55.212671 14989 raft_consensus.cc:2229] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:13:55.213037 14990 raft_consensus.cc:2229] T fa290e1965704676b877617839b5db60 P 6045ad5c346345be95d1a784bdc1b1f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:13:55.213039 14989 raft_consensus.cc:2258] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:13:55.213056 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 14307
I20260812 11:13:55.213994 14991 tablet_replica.cc:330] T fa290e1965704676b877617839b5db60 P 673f4b17487e467fa03b24fb69a0b921: stopping tablet replica
I20260812 11:13:55.216431 14991 raft_consensus.cc:2229] T fa290e1965704676b877617839b5db60 P 673f4b17487e467fa03b24fb69a0b921 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:13:55.216835 14991 raft_consensus.cc:2258] T fa290e1965704676b877617839b5db60 P 673f4b17487e467fa03b24fb69a0b921 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:13:55.217224 14989 ts_tablet_manager.cc:1905] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:13:55.220899 14989 ts_tablet_manager.cc:1918] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:13:55.221105 14989 log.cc:1196] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/fa290e1965704676b877617839b5db60
I20260812 11:13:55.221545 14989 ts_tablet_manager.cc:1939] T fa290e1965704676b877617839b5db60 P 98806d25cf6b4c29b6b49ae7f8f89d5c: Deleting consensus metadata
I20260812 11:13:55.221983 14991 ts_tablet_manager.cc:1905] T fa290e1965704676b877617839b5db60 P 673f4b17487e467fa03b24fb69a0b921: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:13:55.222870 14192 catalog_manager.cc:4841] TS 98806d25cf6b4c29b6b49ae7f8f89d5c (127.17.44.195:45467): tablet fa290e1965704676b877617839b5db60 (table test-table [id=2e5030c118574207920d8a851161d3f8]) successfully deleted
I20260812 11:13:55.224891 14991 ts_tablet_manager.cc:1918] T fa290e1965704676b877617839b5db60 P 673f4b17487e467fa03b24fb69a0b921: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:13:55.224998 14991 log.cc:1196] T fa290e1965704676b877617839b5db60 P 673f4b17487e467fa03b24fb69a0b921: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/fa290e1965704676b877617839b5db60
I20260812 11:13:55.225391 14991 ts_tablet_manager.cc:1939] T fa290e1965704676b877617839b5db60 P 673f4b17487e467fa03b24fb69a0b921: Deleting consensus metadata
I20260812 11:13:55.226610 14193 catalog_manager.cc:4841] TS 673f4b17487e467fa03b24fb69a0b921 (127.17.44.194:34835): tablet fa290e1965704676b877617839b5db60 (table test-table [id=2e5030c118574207920d8a851161d3f8]) successfully deleted
W20260812 11:13:55.229260 14193 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.44.193:34167 (error 108)
I20260812 11:13:55.230217 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 14520
I20260812 11:13:55.230441 14950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:55.230090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:34167 (local address 127.0.0.1:37240)
0812 11:13:55.230243 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:55.230328 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.193:34167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 11:13:55.255793 14193 catalog_manager.cc:4584] TS 6045ad5c346345be95d1a784bdc1b1f6 (127.17.44.193:34167): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fa290e1965704676b877617839b5db60: Network error: Client connection negotiation failed: client connection to 127.17.44.193:34167: connect: Connection refused (error 111)
I20260812 11:13:55.261971 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 14729
I20260812 11:13:55.288061 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 14164
2026-08-12T11:13:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1171 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:13:55 dist-test-slave-drl0 krb5kdc[14998](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:13:55 dist-test-slave-drl0 krb5kdc[14998](info): set up 2 sockets
Aug 12 11:13:55 dist-test-slave-drl0 krb5kdc[14998](info): commencing operation
krb5kdc: starting...
W20260812 11:13:56.608158 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:13:56 dist-test-slave-drl0 krb5kdc[14998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:13:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:13:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:13:56.770895 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:44971
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:35425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:44971
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:13:56.907969 15017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:56.908258 15017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:56.908303 15017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:56.912443 15017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:13:56.912511 15017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:56.912532 15017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:56.912549 15017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:13:56.912564 15017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:13:56.920341 15017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:35425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:44971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:44971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:56.921504 15017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:56.922729 15017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:56.933630 15034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:56.933751 15031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:56.933627 15028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:56.934383 15017 server_base.cc:1034] running on GCE node
I20260812 11:13:56.935115 15017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:56.936530 15017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:56.937760 15017 hybrid_clock.cc:648] HybridClock initialized: now 1786533236937744 us; error 34 us; skew 500 ppm
Aug 12 11:13:56 dist-test-slave-drl0 krb5kdc[14998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533236, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:13:56.941334 15017 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:13:56.942612 15017 webserver.cc:460] Webserver started at http://127.17.44.254:45789/ using document root <none> and password file <none>
I20260812 11:13:56.943306 15017 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:56.943387 15017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:56.943831 15017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:13:56.946173 15017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "76f0c574443648fc99aeaab49bc66415"
format_stamp: "Formatted at 2026-08-12 11:13:56 on dist-test-slave-drl0"
server_key: "53b18206636bc4358388b4a0e27f295c"
server_key_iv: "f1102a33273b1ec7068d9aed9e55e42a"
server_key_version: "encryptionkey@0"
I20260812 11:13:56.946719 15017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "76f0c574443648fc99aeaab49bc66415"
format_stamp: "Formatted at 2026-08-12 11:13:56 on dist-test-slave-drl0"
server_key: "53b18206636bc4358388b4a0e27f295c"
server_key_iv: "f1102a33273b1ec7068d9aed9e55e42a"
server_key_version: "encryptionkey@0"
I20260812 11:13:56.950980 15017 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 11:13:56.953734 15045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:56.955142 15017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:13:56.955241 15017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "76f0c574443648fc99aeaab49bc66415"
format_stamp: "Formatted at 2026-08-12 11:13:56 on dist-test-slave-drl0"
server_key: "53b18206636bc4358388b4a0e27f295c"
server_key_iv: "f1102a33273b1ec7068d9aed9e55e42a"
server_key_version: "encryptionkey@0"
I20260812 11:13:56.955329 15017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:56.973192 15017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:56.976977 15017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:56.977150 15017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:56.985870 15017 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:44971
I20260812 11:13:56.985904 15128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:44971 every 8 connection(s)
I20260812 11:13:56.987593 15017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:13:56.990758 15129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:56.991886 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 15017
I20260812 11:13:56.992058 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:13:56.992367 17587 external_mini_cluster.cc:1428] Setting key 799ba82c4941ee1fa9a29e8ac8550376
I20260812 11:13:57.000908 15129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76f0c574443648fc99aeaab49bc66415: Bootstrap starting.
Aug 12 11:13:57 dist-test-slave-drl0 krb5kdc[14998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:13:57.004439 15129 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76f0c574443648fc99aeaab49bc66415: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:57.005554 15129 log.cc:824] T 00000000000000000000000000000000 P 76f0c574443648fc99aeaab49bc66415: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:57.008237 15129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76f0c574443648fc99aeaab49bc66415: No bootstrap required, opened a new log
I20260812 11:13:57.010648 15136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:56.994035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48352 (local address 127.17.44.254:44971)
0812 11:13:56.994802 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:56.994830 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:56.994877 (+ 47us) server_negotiation.cc:407] Connection header received
0812 11:13:56.995896 (+ 1019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:56.995922 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:56.996353 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:56.996745 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:56.997429 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:56.998998 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:56.999923 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.000770 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.001055 (+ 285us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:57.004153 (+ 3098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:57.004160 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:57.004178 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:13:57.004219 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:57.006660 (+ 2441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:57.007460 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:57.007466 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:57.007472 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:57.007626 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:57.008098 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:57.008102 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:57.008104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:57.008669 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:57.008925 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:57.009219 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 11:13:57.009656 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":254,"threads_started":1}
I20260812 11:13:57.013521 15129 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76f0c574443648fc99aeaab49bc66415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76f0c574443648fc99aeaab49bc66415" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44971 } }
I20260812 11:13:57.013757 15129 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76f0c574443648fc99aeaab49bc66415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:57.013804 15129 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76f0c574443648fc99aeaab49bc66415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76f0c574443648fc99aeaab49bc66415, State: Initialized, Role: FOLLOWER
I20260812 11:13:57.014168 15129 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76f0c574443648fc99aeaab49bc66415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76f0c574443648fc99aeaab49bc66415" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44971 } }
I20260812 11:13:57.014281 15129 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76f0c574443648fc99aeaab49bc66415 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:13:57.014317 15129 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76f0c574443648fc99aeaab49bc66415 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:13:57.014384 15129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76f0c574443648fc99aeaab49bc66415 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:57.015383 15129 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76f0c574443648fc99aeaab49bc66415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76f0c574443648fc99aeaab49bc66415" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44971 } }
I20260812 11:13:57.015733 15129 leader_election.cc:302] T 00000000000000000000000000000000 P 76f0c574443648fc99aeaab49bc66415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76f0c574443648fc99aeaab49bc66415; no voters:
I20260812 11:13:57.016144 15129 leader_election.cc:288] T 00000000000000000000000000000000 P 76f0c574443648fc99aeaab49bc66415 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:13:57.016288 15139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76f0c574443648fc99aeaab49bc66415 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:13:57.016559 15139 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76f0c574443648fc99aeaab49bc66415 [term 1 LEADER]: Becoming Leader. State: Replica: 76f0c574443648fc99aeaab49bc66415, State: Running, Role: LEADER
I20260812 11:13:57.016899 15139 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76f0c574443648fc99aeaab49bc66415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76f0c574443648fc99aeaab49bc66415" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44971 } }
I20260812 11:13:57.018181 15139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76f0c574443648fc99aeaab49bc66415 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76f0c574443648fc99aeaab49bc66415. Latest consensus state: current_term: 1 leader_uuid: "76f0c574443648fc99aeaab49bc66415" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76f0c574443648fc99aeaab49bc66415" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44971 } } }
I20260812 11:13:57.018324 15139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76f0c574443648fc99aeaab49bc66415 [sys.catalog]: This master's current role is: LEADER
I20260812 11:13:57.018441 15129 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76f0c574443648fc99aeaab49bc66415 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:13:57.018618 15140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76f0c574443648fc99aeaab49bc66415 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76f0c574443648fc99aeaab49bc66415" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76f0c574443648fc99aeaab49bc66415" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44971 } } }
I20260812 11:13:57.018746 15140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76f0c574443648fc99aeaab49bc66415 [sys.catalog]: This master's current role is: LEADER
I20260812 11:13:57.019182 15145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:13:57.022616 15145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:13:57.030831 15145 catalog_manager.cc:1347] Generated new cluster ID: 55c8e04a47234aeb91fccd6ea3075e94
I20260812 11:13:57.030920 15145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:13:57.049676 15145 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:13:57.051565 15145 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:13:57.060096 15145 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76f0c574443648fc99aeaab49bc66415: Generated new TSK 0
I20260812 11:13:57.060791 15145 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
Entry for principal kudu/127.17.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab.
I20260812 11:13:57.103461 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:44971
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:35425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:13:57.224818 15171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:57.225068 15171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:57.225111 15171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:57.229166 15171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:57.229264 15171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:57.229360 15171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:13:57.234763 15171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:35425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.193.keytab
--principal=kudu/127.17.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:44971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:57.235852 15171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:57.236886 15171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:57.247609 15180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:57.247640 15181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:57.247752 15185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:57.248067 15171 server_base.cc:1034] running on GCE node
I20260812 11:13:57.248701 15171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:57.249579 15171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:57.250836 15171 hybrid_clock.cc:648] HybridClock initialized: now 1786533237250794 us; error 51 us; skew 500 ppm
Aug 12 11:13:57 dist-test-slave-drl0 krb5kdc[14998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533237, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:13:57.254041 15171 init.cc:377] Logged in from keytab as kudu/127.17.44.193@KRBTEST.COM (short username kudu)
I20260812 11:13:57.255014 15171 webserver.cc:460] Webserver started at http://127.17.44.193:43121/ using document root <none> and password file <none>
I20260812 11:13:57.255599 15171 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:57.255662 15171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:57.255872 15171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:13:57.257887 15171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "d079a46cbb004d438f1ddf3c4886090b"
format_stamp: "Formatted at 2026-08-12 11:13:57 on dist-test-slave-drl0"
server_key: "fee91f7c6e2be50a885c0692cd60edfd"
server_key_iv: "0f26f6086aca399e511fce7ad9afd49f"
server_key_version: "encryptionkey@0"
I20260812 11:13:57.258474 15171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "d079a46cbb004d438f1ddf3c4886090b"
format_stamp: "Formatted at 2026-08-12 11:13:57 on dist-test-slave-drl0"
server_key: "fee91f7c6e2be50a885c0692cd60edfd"
server_key_iv: "0f26f6086aca399e511fce7ad9afd49f"
server_key_version: "encryptionkey@0"
I20260812 11:13:57.262774 15171 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 11:13:57.265507 15197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:57.267099 15171 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:13:57.267203 15171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "d079a46cbb004d438f1ddf3c4886090b"
format_stamp: "Formatted at 2026-08-12 11:13:57 on dist-test-slave-drl0"
server_key: "fee91f7c6e2be50a885c0692cd60edfd"
server_key_iv: "0f26f6086aca399e511fce7ad9afd49f"
server_key_version: "encryptionkey@0"
I20260812 11:13:57.267331 15171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:57.296895 15171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:57.300295 15171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:57.300513 15171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:57.301203 15171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:13:57.302078 15171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:13:57.302132 15171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:57.302173 15171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:13:57.302189 15171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:57.313190 15171 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:33237
I20260812 11:13:57.313222 15363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:33237 every 8 connection(s)
I20260812 11:13:57.314709 15171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:13:57.321628 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 15171
I20260812 11:13:57.321759 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:13:57.322240 17587 external_mini_cluster.cc:1428] Setting key d4c335564401cf20a2762cb8e74ac7d7
Aug 12 11:13:57 dist-test-slave-drl0 krb5kdc[14998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533237, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.193@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:13:57.332047 15136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:57.317060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:49757 (local address 127.17.44.254:44971)
0812 11:13:57.317260 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:57.317264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:57.318674 (+ 1410us) server_negotiation.cc:407] Connection header received
0812 11:13:57.319614 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:57.319617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:57.319674 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:57.319783 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:57.321017 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.321672 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.323756 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.324117 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.324239 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:57.327234 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:57.327240 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:57.327244 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:13:57.327359 (+ 115us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:57.329377 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:57.330036 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:57.330043 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:57.330048 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:57.330150 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:57.330617 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:57.330621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:57.330623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:57.330889 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:57.331045 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:57.331630 (+ 585us) server_negotiation.cc:299] Negotiation successful
0812 11:13:57.331800 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260812 11:13:57.332676 15370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:57.317411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:44971 (local address 127.17.44.193:49757)
0812 11:13:57.318523 (+ 1112us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:57.318566 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:57.319374 (+ 808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:57.319976 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:57.319982 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:57.320370 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:13:57.320854 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:57.320867 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.321973 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.321978 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:57.323593 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:57.323605 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.324284 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.324288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:57.324469 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:57.325085 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:13:57.325101 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:13:57.327054 (+ 1953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:13:57.329604 (+ 2550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:57.329609 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:57.329622 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:57.329894 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:57.330316 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:57.330320 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:57.330322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:57.330493 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:57.331085 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:13:57.331091 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:13:57.331345 (+ 254us) client_negotiation.cc:769] Sending connection context
0812 11:13:57.331512 (+ 167us) client_negotiation.cc:240] Negotiation successful
0812 11:13:57.331786 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"thread_start_us":564,"threads_started":1}
I20260812 11:13:57.334054 15365 heartbeater.cc:347] Connected to a master server at 127.17.44.254:44971
I20260812 11:13:57.334342 15365 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.17.44.194@KRBTEST.COM; defaulting to no policy
I20260812 11:13:57.335031 15365 heartbeater.cc:511] Master 127.17.44.254:44971 requested a full tablet report, sending...
I20260812 11:13:57.336586 15072 ts_manager.cc:194] Registered new tserver with Master: d079a46cbb004d438f1ddf3c4886090b (127.17.44.193:33237)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.194@KRBTEST.COM" created.
I20260812 11:13:57.338017 15072 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.193@KRBTEST.COM'} at 127.17.44.193:49757
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
Entry for principal kudu/127.17.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab.
I20260812 11:13:57.355979 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:44971
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:35425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:13:57.479024 15373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:57.479308 15373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:57.479385 15373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:57.483577 15373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:57.483661 15373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:57.483750 15373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:13:57.489339 15373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:35425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.194.keytab
--principal=kudu/127.17.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:44971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:57.490368 15373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:57.491449 15373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:57.501510 15387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:57.501580 15373 server_base.cc:1034] running on GCE node
W20260812 11:13:57.501533 15385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:57.501524 15390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:57.502051 15373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:57.502768 15373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:57.503968 15373 hybrid_clock.cc:648] HybridClock initialized: now 1786533237503937 us; error 42 us; skew 500 ppm
Aug 12 11:13:57 dist-test-slave-drl0 krb5kdc[14998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533237, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:13:57.507432 15373 init.cc:377] Logged in from keytab as kudu/127.17.44.194@KRBTEST.COM (short username kudu)
I20260812 11:13:57.508301 15373 webserver.cc:460] Webserver started at http://127.17.44.194:45607/ using document root <none> and password file <none>
I20260812 11:13:57.508878 15373 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:57.508981 15373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:57.509215 15373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:13:57.511834 15373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "9deca17b4a7d46c5aadf4ecf88d21171"
format_stamp: "Formatted at 2026-08-12 11:13:57 on dist-test-slave-drl0"
server_key: "fe8d6552c5b9c1bd241a405653a99786"
server_key_iv: "8e257b3e1ffd376f393104141182a54a"
server_key_version: "encryptionkey@0"
I20260812 11:13:57.512422 15373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "9deca17b4a7d46c5aadf4ecf88d21171"
format_stamp: "Formatted at 2026-08-12 11:13:57 on dist-test-slave-drl0"
server_key: "fe8d6552c5b9c1bd241a405653a99786"
server_key_iv: "8e257b3e1ffd376f393104141182a54a"
server_key_version: "encryptionkey@0"
I20260812 11:13:57.516351 15373 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 11:13:57.518850 15399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:57.520377 15373 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 11:13:57.520540 15373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "9deca17b4a7d46c5aadf4ecf88d21171"
format_stamp: "Formatted at 2026-08-12 11:13:57 on dist-test-slave-drl0"
server_key: "fe8d6552c5b9c1bd241a405653a99786"
server_key_iv: "8e257b3e1ffd376f393104141182a54a"
server_key_version: "encryptionkey@0"
I20260812 11:13:57.520653 15373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:57.536486 15373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:57.539970 15373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:57.540182 15373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:57.540809 15373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:13:57.541682 15373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:13:57.541759 15373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:57.541827 15373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:13:57.541860 15373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:57.552683 15373 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:46709
I20260812 11:13:57.552752 15575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:46709 every 8 connection(s)
I20260812 11:13:57.553654 15373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:13:57.555289 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 15373
I20260812 11:13:57.555430 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:13:57.555821 17587 external_mini_cluster.cc:1428] Setting key d4a74f78ef93eb970e306a7c7983bdac
Aug 12 11:13:57 dist-test-slave-drl0 krb5kdc[14998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533237, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.194@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.44.195@KRBTEST.COM; defaulting to no policy
I20260812 11:13:57.569576 15136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:57.555735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:58559 (local address 127.17.44.254:44971)
0812 11:13:57.555914 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:57.555919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:57.556903 (+ 984us) server_negotiation.cc:407] Connection header received
0812 11:13:57.558411 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:57.558414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:57.558468 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:57.558579 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:57.560111 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.560663 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.561838 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.562079 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.562193 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:57.564910 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:57.564916 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:57.564922 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:13:57.564959 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:57.566795 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:57.567401 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:57.567408 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:57.567412 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:57.567510 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:57.568017 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:57.568024 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:57.568027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:57.568629 (+ 602us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:57.568767 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:57.569342 (+ 575us) server_negotiation.cc:299] Negotiation successful
0812 11:13:57.569451 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 11:13:57.570521 15578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:57.556146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:44971 (local address 127.17.44.194:58559)
0812 11:13:57.556749 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:57.556791 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:57.558159 (+ 1368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:57.558763 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:57.558769 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:57.559304 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:13:57.559966 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:57.559983 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.560793 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.560796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:57.561695 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:57.561711 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.562233 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.562237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:57.562409 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:57.563156 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:13:57.563182 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:13:57.564724 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:13:57.567002 (+ 2278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:57.567010 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:57.567021 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:57.567277 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:57.567639 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:57.567641 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:57.567643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:57.567787 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:57.568793 (+ 1006us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:13:57.568799 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:13:57.569027 (+ 228us) client_negotiation.cc:769] Sending connection context
0812 11:13:57.569188 (+ 161us) client_negotiation.cc:240] Negotiation successful
0812 11:13:57.569562 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":173,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.195@KRBTEST.COM" created.
I20260812 11:13:57.572358 15576 heartbeater.cc:347] Connected to a master server at 127.17.44.254:44971
I20260812 11:13:57.572620 15576 heartbeater.cc:464] Registering TS with master...
I20260812 11:13:57.573108 15576 heartbeater.cc:511] Master 127.17.44.254:44971 requested a full tablet report, sending...
I20260812 11:13:57.574229 15072 ts_manager.cc:194] Registered new tserver with Master: 9deca17b4a7d46c5aadf4ecf88d21171 (127.17.44.194:46709)
I20260812 11:13:57.574822 15072 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.194@KRBTEST.COM'} at 127.17.44.194:58559
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
Entry for principal kudu/127.17.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab.
I20260812 11:13:57.590329 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:44971
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:35425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:13:57.708662 15583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:57.709026 15583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:57.709074 15583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:57.713754 15583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:57.713894 15583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:57.714005 15583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:13:57.719558 15583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:35425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.195.keytab
--principal=kudu/127.17.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:44971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:57.720659 15583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:57.721681 15583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:57.731534 15592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:57.731575 15594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:57.731719 15591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:57.732465 15583 server_base.cc:1034] running on GCE node
I20260812 11:13:57.732872 15583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:57.733469 15583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:57.734679 15583 hybrid_clock.cc:648] HybridClock initialized: now 1786533237734641 us; error 56 us; skew 500 ppm
Aug 12 11:13:57 dist-test-slave-drl0 krb5kdc[14998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533237, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:13:57.737709 15583 init.cc:377] Logged in from keytab as kudu/127.17.44.195@KRBTEST.COM (short username kudu)
I20260812 11:13:57.738502 15583 webserver.cc:460] Webserver started at http://127.17.44.195:46153/ using document root <none> and password file <none>
I20260812 11:13:57.739087 15583 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:57.739156 15583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:57.739326 15583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:13:57.741297 15583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "4fc803d2c9b34e5881f98d2ccc5b6d82"
format_stamp: "Formatted at 2026-08-12 11:13:57 on dist-test-slave-drl0"
server_key: "fe0ad31edc1f26702ede7f9450af66bb"
server_key_iv: "9468bd3edf350c628297557e971ba8a3"
server_key_version: "encryptionkey@0"
I20260812 11:13:57.741813 15583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "4fc803d2c9b34e5881f98d2ccc5b6d82"
format_stamp: "Formatted at 2026-08-12 11:13:57 on dist-test-slave-drl0"
server_key: "fe0ad31edc1f26702ede7f9450af66bb"
server_key_iv: "9468bd3edf350c628297557e971ba8a3"
server_key_version: "encryptionkey@0"
I20260812 11:13:57.745676 15583 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 11:13:57.748220 15604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:57.749449 15583 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 11:13:57.749549 15583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "4fc803d2c9b34e5881f98d2ccc5b6d82"
format_stamp: "Formatted at 2026-08-12 11:13:57 on dist-test-slave-drl0"
server_key: "fe0ad31edc1f26702ede7f9450af66bb"
server_key_iv: "9468bd3edf350c628297557e971ba8a3"
server_key_version: "encryptionkey@0"
I20260812 11:13:57.749634 15583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:57.774338 15583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:57.777769 15583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:57.777992 15583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:57.778618 15583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:13:57.779659 15583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:13:57.779708 15583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:57.779749 15583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:13:57.779765 15583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:57.790350 15583 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:46217
I20260812 11:13:57.790442 15785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:46217 every 8 connection(s)
I20260812 11:13:57.791352 15583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:13:57.798071 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 15583
I20260812 11:13:57.798224 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:13:57.798566 17587 external_mini_cluster.cc:1428] Setting key d420f934f6350c5a04f455be7a854c91
Aug 12 11:13:57 dist-test-slave-drl0 krb5kdc[14998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533237, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.195@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:13:57.807034 15136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:57.793376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:44061 (local address 127.17.44.254:44971)
0812 11:13:57.793536 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:57.793541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:57.794429 (+ 888us) server_negotiation.cc:407] Connection header received
0812 11:13:57.795725 (+ 1296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:57.795728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:57.795859 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:57.795947 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:57.797086 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.797635 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.799371 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.799706 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.799818 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:57.802854 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:57.802859 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:57.802862 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:13:57.802907 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:57.804584 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:57.805127 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:57.805130 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:57.805132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:57.805308 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:57.805816 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:57.805819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:57.805820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:57.806033 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:57.806130 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:57.806750 (+ 620us) server_negotiation.cc:299] Negotiation successful
0812 11:13:57.806867 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 11:13:57.807893 15793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:57.793750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:44971 (local address 127.17.44.195:44061)
0812 11:13:57.794221 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:57.794262 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:57.795489 (+ 1227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:57.796104 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:57.796111 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:57.796454 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:13:57.796918 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:57.796930 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.797837 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.797843 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:57.799220 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:57.799230 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.799895 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.799900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:57.800101 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:57.800770 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:13:57.800804 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:13:57.802577 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:13:57.804740 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:57.804744 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:57.804757 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:57.805011 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:57.805526 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:13:57.805530 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:13:57.805531 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:13:57.805680 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:13:57.806215 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:13:57.806221 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:13:57.806462 (+ 241us) client_negotiation.cc:769] Sending connection context
0812 11:13:57.806638 (+ 176us) client_negotiation.cc:240] Negotiation successful
0812 11:13:57.806915 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":122,"threads_started":1}
I20260812 11:13:57.809269 15787 heartbeater.cc:347] Connected to a master server at 127.17.44.254:44971
I20260812 11:13:57.809724 15787 heartbeater.cc:464] Registering TS with master...
I20260812 11:13:57.810534 15787 heartbeater.cc:511] Master 127.17.44.254:44971 requested a full tablet report, sending...
I20260812 11:13:57.812212 15069 ts_manager.cc:194] Registered new tserver with Master: 4fc803d2c9b34e5881f98d2ccc5b6d82 (127.17.44.195:46217)
I20260812 11:13:57.813009 15069 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.44.195@KRBTEST.COM'} at 127.17.44.195:44061
I20260812 11:13:57.813337 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:13:57.825093 15136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:57.815759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48360 (local address 127.17.44.254:44971)
0812 11:13:57.815909 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:57.815913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:57.816177 (+ 264us) server_negotiation.cc:407] Connection header received
0812 11:13:57.816311 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:57.816313 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:57.816352 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:57.816420 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:57.817514 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.818012 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.818996 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.819229 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.819473 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:57.821095 (+ 1622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:57.821098 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:57.821100 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:13:57.821121 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:57.822817 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:57.823345 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:57.823349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:57.823351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:57.823509 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:13:57.823958 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:13:57.823962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:13:57.823963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:13:57.824197 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:57.824354 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:57.824722 (+ 368us) server_negotiation.cc:299] Negotiation successful
0812 11:13:57.824849 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 11:13:57.828487 15069 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48360:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:13:57.831875 15069 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:13:57.847699 15812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:57.840815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41260 (local address 127.17.44.195:46217)
0812 11:13:57.841644 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:57.841659 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:57.841685 (+ 26us) server_negotiation.cc:407] Connection header received
0812 11:13:57.841847 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:57.841864 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:57.842095 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:57.842727 (+ 632us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:57.842875 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.844055 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.846082 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.846800 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.847127 (+ 327us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:57.847298 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:57.847437 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 11:13:57.847533 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":708,"thread_start_us":122,"threads_started":1}
I20260812 11:13:57.847683 15811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:57.840668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:46217 (local address 127.0.0.1:41260)
0812 11:13:57.841101 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:57.841146 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:57.841265 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:57.842313 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:57.842322 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:57.842354 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:57.842563 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:57.842575 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.844210 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.844213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:57.845984 (+ 1771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:57.845993 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.847119 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.847123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:57.847246 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:57.847286 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 11:13:57.847407 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 11:13:57.847531 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":389,"thread_start_us":72,"threads_started":1}
I20260812 11:13:57.848102 15810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:57.840708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:33237 (local address 127.0.0.1:58120)
0812 11:13:57.841789 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:57.841816 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:57.841912 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:57.842830 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:57.842834 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:57.842856 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:57.842996 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:57.843006 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.844164 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.844168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:57.845886 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:57.845899 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.847463 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.847468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:57.847626 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:57.847717 (+ 91us) client_negotiation.cc:769] Sending connection context
0812 11:13:57.847943 (+ 226us) client_negotiation.cc:240] Negotiation successful
0812 11:13:57.848004 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1011,"mutex_wait_us":164,"thread_start_us":114,"threads_started":1}
I20260812 11:13:57.848620 15816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:57.841763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58120 (local address 127.17.44.193:33237)
0812 11:13:57.842163 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:57.842171 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:57.842193 (+ 22us) server_negotiation.cc:407] Connection header received
0812 11:13:57.842269 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:57.842273 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:57.842468 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:57.842875 (+ 407us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:57.843113 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.843987 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.846027 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.846975 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.847087 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:57.847379 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:57.848349 (+ 970us) server_negotiation.cc:299] Negotiation successful
0812 11:13:57.848467 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":92,"threads_started":1}
I20260812 11:13:57.848794 15814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:57.841537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:46709 (local address 127.0.0.1:57674)
0812 11:13:57.842275 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:57.842288 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:57.842372 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:57.843406 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:57.843409 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:57.843429 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:57.843525 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:57.843531 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.844759 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.844764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:57.845850 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:57.845860 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.847463 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.847468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:57.847628 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:57.847721 (+ 93us) client_negotiation.cc:769] Sending connection context
0812 11:13:57.847799 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 11:13:57.847891 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"mutex_wait_us":72,"thread_start_us":103,"threads_started":1}
I20260812 11:13:57.849077 15819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:57.842243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57674 (local address 127.17.44.194:46709)
0812 11:13:57.842977 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:57.842989 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:57.843010 (+ 21us) server_negotiation.cc:407] Connection header received
0812 11:13:57.843082 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:57.843090 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:57.843258 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:57.843406 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:57.843650 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.844625 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.845975 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:57.846974 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:57.847087 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:57.847379 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:57.848866 (+ 1487us) server_negotiation.cc:299] Negotiation successful
0812 11:13:57.848975 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":125,"threads_started":1}
I20260812 11:13:57.850821 15684 tablet_service.cc:1467] Processing CreateTablet for tablet 673f41d0ccba434b8b9ca6c3aa353ccb (DEFAULT_TABLE table=test-table [id=ba03124008e04898affe784041466e6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:13:57.851543 15267 tablet_service.cc:1467] Processing CreateTablet for tablet 673f41d0ccba434b8b9ca6c3aa353ccb (DEFAULT_TABLE table=test-table [id=ba03124008e04898affe784041466e6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:13:57.852035 15684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 673f41d0ccba434b8b9ca6c3aa353ccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:57.851684 15474 tablet_service.cc:1467] Processing CreateTablet for tablet 673f41d0ccba434b8b9ca6c3aa353ccb (DEFAULT_TABLE table=test-table [id=ba03124008e04898affe784041466e6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:13:57.852475 15267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 673f41d0ccba434b8b9ca6c3aa353ccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:57.852721 15474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 673f41d0ccba434b8b9ca6c3aa353ccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:57.859261 15820 tablet_bootstrap.cc:492] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171: Bootstrap starting.
I20260812 11:13:57.860438 15821 tablet_bootstrap.cc:492] T 673f41d0ccba434b8b9ca6c3aa353ccb P d079a46cbb004d438f1ddf3c4886090b: Bootstrap starting.
I20260812 11:13:57.862205 15820 tablet_bootstrap.cc:654] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:57.862623 15821 tablet_bootstrap.cc:654] T 673f41d0ccba434b8b9ca6c3aa353ccb P d079a46cbb004d438f1ddf3c4886090b: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:57.863422 15823 tablet_bootstrap.cc:492] T 673f41d0ccba434b8b9ca6c3aa353ccb P 4fc803d2c9b34e5881f98d2ccc5b6d82: Bootstrap starting.
I20260812 11:13:57.863649 15821 log.cc:824] T 673f41d0ccba434b8b9ca6c3aa353ccb P d079a46cbb004d438f1ddf3c4886090b: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:57.863649 15820 log.cc:824] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:57.865612 15823 tablet_bootstrap.cc:654] T 673f41d0ccba434b8b9ca6c3aa353ccb P 4fc803d2c9b34e5881f98d2ccc5b6d82: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:57.866197 15820 tablet_bootstrap.cc:492] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171: No bootstrap required, opened a new log
I20260812 11:13:57.866371 15821 tablet_bootstrap.cc:492] T 673f41d0ccba434b8b9ca6c3aa353ccb P d079a46cbb004d438f1ddf3c4886090b: No bootstrap required, opened a new log
I20260812 11:13:57.866444 15820 ts_tablet_manager.cc:1397] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 11:13:57.866575 15821 ts_tablet_manager.cc:1397] T 673f41d0ccba434b8b9ca6c3aa353ccb P d079a46cbb004d438f1ddf3c4886090b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 11:13:57.866972 15823 log.cc:824] T 673f41d0ccba434b8b9ca6c3aa353ccb P 4fc803d2c9b34e5881f98d2ccc5b6d82: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:57.869082 15823 tablet_bootstrap.cc:492] T 673f41d0ccba434b8b9ca6c3aa353ccb P 4fc803d2c9b34e5881f98d2ccc5b6d82: No bootstrap required, opened a new log
I20260812 11:13:57.869268 15823 ts_tablet_manager.cc:1397] T 673f41d0ccba434b8b9ca6c3aa353ccb P 4fc803d2c9b34e5881f98d2ccc5b6d82: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 11:13:57.869812 15820 raft_consensus.cc:348] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc803d2c9b34e5881f98d2ccc5b6d82" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46217 } } peers { permanent_uuid: "d079a46cbb004d438f1ddf3c4886090b" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 33237 } } peers { permanent_uuid: "9deca17b4a7d46c5aadf4ecf88d21171" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46709 } }
I20260812 11:13:57.870095 15820 raft_consensus.cc:374] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:57.870152 15820 raft_consensus.cc:729] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9deca17b4a7d46c5aadf4ecf88d21171, State: Initialized, Role: FOLLOWER
I20260812 11:13:57.870672 15820 consensus_queue.cc:260] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc803d2c9b34e5881f98d2ccc5b6d82" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46217 } } peers { permanent_uuid: "d079a46cbb004d438f1ddf3c4886090b" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 33237 } } peers { permanent_uuid: "9deca17b4a7d46c5aadf4ecf88d21171" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46709 } }
I20260812 11:13:57.871474 15576 heartbeater.cc:503] Master 127.17.44.254:44971 was elected leader, sending a full tablet report...
I20260812 11:13:57.871804 15820 ts_tablet_manager.cc:1428] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260812 11:13:57.872395 15821 raft_consensus.cc:348] T 673f41d0ccba434b8b9ca6c3aa353ccb P d079a46cbb004d438f1ddf3c4886090b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc803d2c9b34e5881f98d2ccc5b6d82" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46217 } } peers { permanent_uuid: "d079a46cbb004d438f1ddf3c4886090b" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 33237 } } peers { permanent_uuid: "9deca17b4a7d46c5aadf4ecf88d21171" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46709 } }
I20260812 11:13:57.873224 15823 raft_consensus.cc:348] T 673f41d0ccba434b8b9ca6c3aa353ccb P 4fc803d2c9b34e5881f98d2ccc5b6d82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc803d2c9b34e5881f98d2ccc5b6d82" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46217 } } peers { permanent_uuid: "d079a46cbb004d438f1ddf3c4886090b" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 33237 } } peers { permanent_uuid: "9deca17b4a7d46c5aadf4ecf88d21171" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46709 } }
I20260812 11:13:57.873461 15823 raft_consensus.cc:374] T 673f41d0ccba434b8b9ca6c3aa353ccb P 4fc803d2c9b34e5881f98d2ccc5b6d82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:57.873512 15823 raft_consensus.cc:729] T 673f41d0ccba434b8b9ca6c3aa353ccb P 4fc803d2c9b34e5881f98d2ccc5b6d82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fc803d2c9b34e5881f98d2ccc5b6d82, State: Initialized, Role: FOLLOWER
I20260812 11:13:57.873956 15823 consensus_queue.cc:260] T 673f41d0ccba434b8b9ca6c3aa353ccb P 4fc803d2c9b34e5881f98d2ccc5b6d82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc803d2c9b34e5881f98d2ccc5b6d82" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46217 } } peers { permanent_uuid: "d079a46cbb004d438f1ddf3c4886090b" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 33237 } } peers { permanent_uuid: "9deca17b4a7d46c5aadf4ecf88d21171" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46709 } }
I20260812 11:13:57.874838 15787 heartbeater.cc:503] Master 127.17.44.254:44971 was elected leader, sending a full tablet report...
I20260812 11:13:57.874945 15823 ts_tablet_manager.cc:1428] T 673f41d0ccba434b8b9ca6c3aa353ccb P 4fc803d2c9b34e5881f98d2ccc5b6d82: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260812 11:13:57.875345 15821 raft_consensus.cc:374] T 673f41d0ccba434b8b9ca6c3aa353ccb P d079a46cbb004d438f1ddf3c4886090b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:57.875437 15821 raft_consensus.cc:729] T 673f41d0ccba434b8b9ca6c3aa353ccb P d079a46cbb004d438f1ddf3c4886090b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d079a46cbb004d438f1ddf3c4886090b, State: Initialized, Role: FOLLOWER
I20260812 11:13:57.876021 15821 consensus_queue.cc:260] T 673f41d0ccba434b8b9ca6c3aa353ccb P d079a46cbb004d438f1ddf3c4886090b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc803d2c9b34e5881f98d2ccc5b6d82" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46217 } } peers { permanent_uuid: "d079a46cbb004d438f1ddf3c4886090b" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 33237 } } peers { permanent_uuid: "9deca17b4a7d46c5aadf4ecf88d21171" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46709 } }
I20260812 11:13:57.877074 15821 ts_tablet_manager.cc:1428] T 673f41d0ccba434b8b9ca6c3aa353ccb P d079a46cbb004d438f1ddf3c4886090b: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20260812 11:13:57.877408 15365 heartbeater.cc:503] Master 127.17.44.254:44971 was elected leader, sending a full tablet report...
I20260812 11:13:58.035619 15829 raft_consensus.cc:482] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:13:58.035924 15829 raft_consensus.cc:504] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc803d2c9b34e5881f98d2ccc5b6d82" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46217 } } peers { permanent_uuid: "d079a46cbb004d438f1ddf3c4886090b" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 33237 } } peers { permanent_uuid: "9deca17b4a7d46c5aadf4ecf88d21171" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46709 } }
I20260812 11:13:58.037042 15829 leader_election.cc:288] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4fc803d2c9b34e5881f98d2ccc5b6d82 (127.17.44.195:46217), d079a46cbb004d438f1ddf3c4886090b (127.17.44.193:33237)
I20260812 11:13:58.040884 15812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:58.037416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:54109 (local address 127.17.44.195:46217)
0812 11:13:58.037546 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:58.037552 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:58.037629 (+ 77us) server_negotiation.cc:407] Connection header received
0812 11:13:58.037814 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:58.037817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:58.037879 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:58.037965 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:58.038258 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:58.038785 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:58.039898 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:58.040290 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:58.040419 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:58.040476 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:58.040687 (+ 211us) server_negotiation.cc:299] Negotiation successful
0812 11:13:58.040755 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 11:13:58.040882 15578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:58.037319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:46217 (local address 127.17.44.194:54109)
0812 11:13:58.037544 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:58.037560 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:58.037667 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:58.037981 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:58.037984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:58.038002 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:58.038132 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:58.038139 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:58.038907 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:58.038913 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:58.039754 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:58.039763 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:58.040415 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:58.040418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:58.040521 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:58.040581 (+ 60us) client_negotiation.cc:769] Sending connection context
0812 11:13:58.040669 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 11:13:58.040744 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":32}
I20260812 11:13:58.040999 15816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:58.037422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:49745 (local address 127.17.44.193:33237)
0812 11:13:58.037549 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:58.037553 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:58.037816 (+ 263us) server_negotiation.cc:407] Connection header received
0812 11:13:58.037977 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:58.037981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:58.038029 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:58.038122 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:58.038397 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:58.038895 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:58.039981 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:58.040396 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:58.040536 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:58.040591 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:58.040784 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 11:13:58.040857 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 11:13:58.041024 15838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:58.037319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:33237 (local address 127.17.44.194:49745)
0812 11:13:58.037713 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:58.037725 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:58.037857 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:58.038134 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:58.038136 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:58.038153 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:58.038287 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:58.038292 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:58.039075 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:58.039079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:58.039846 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:58.039855 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:58.040531 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:58.040535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:58.040661 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:58.040679 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 11:13:58.040767 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 11:13:58.040834 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":41,"thread_start_us":111,"threads_started":1}
I20260812 11:13:58.041644 15713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "673f41d0ccba434b8b9ca6c3aa353ccb" candidate_uuid: "9deca17b4a7d46c5aadf4ecf88d21171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fc803d2c9b34e5881f98d2ccc5b6d82" is_pre_election: true
I20260812 11:13:58.041982 15713 raft_consensus.cc:2454] T 673f41d0ccba434b8b9ca6c3aa353ccb P 4fc803d2c9b34e5881f98d2ccc5b6d82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9deca17b4a7d46c5aadf4ecf88d21171 in term 0.
I20260812 11:13:58.041997 15296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "673f41d0ccba434b8b9ca6c3aa353ccb" candidate_uuid: "9deca17b4a7d46c5aadf4ecf88d21171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d079a46cbb004d438f1ddf3c4886090b" is_pre_election: true
I20260812 11:13:58.042253 15296 raft_consensus.cc:2454] T 673f41d0ccba434b8b9ca6c3aa353ccb P d079a46cbb004d438f1ddf3c4886090b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9deca17b4a7d46c5aadf4ecf88d21171 in term 0.
W20260812 11:13:58.042459 15790 tablet.cc:2369] T 673f41d0ccba434b8b9ca6c3aa353ccb P 4fc803d2c9b34e5881f98d2ccc5b6d82: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:13:58.042876 15402 leader_election.cc:302] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4fc803d2c9b34e5881f98d2ccc5b6d82, 9deca17b4a7d46c5aadf4ecf88d21171, d079a46cbb004d438f1ddf3c4886090b; no voters:
I20260812 11:13:58.043285 15829 raft_consensus.cc:2781] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:13:58.043416 15829 raft_consensus.cc:482] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:13:58.043450 15829 raft_consensus.cc:3037] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:58.044873 15829 raft_consensus.cc:504] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc803d2c9b34e5881f98d2ccc5b6d82" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46217 } } peers { permanent_uuid: "d079a46cbb004d438f1ddf3c4886090b" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 33237 } } peers { permanent_uuid: "9deca17b4a7d46c5aadf4ecf88d21171" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46709 } }
I20260812 11:13:58.045418 15829 leader_election.cc:288] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171 [CANDIDATE]: Term 1 election: Requested vote from peers 4fc803d2c9b34e5881f98d2ccc5b6d82 (127.17.44.195:46217), d079a46cbb004d438f1ddf3c4886090b (127.17.44.193:33237)
I20260812 11:13:58.045912 15713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "673f41d0ccba434b8b9ca6c3aa353ccb" candidate_uuid: "9deca17b4a7d46c5aadf4ecf88d21171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fc803d2c9b34e5881f98d2ccc5b6d82"
I20260812 11:13:58.046000 15296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "673f41d0ccba434b8b9ca6c3aa353ccb" candidate_uuid: "9deca17b4a7d46c5aadf4ecf88d21171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d079a46cbb004d438f1ddf3c4886090b"
I20260812 11:13:58.046053 15713 raft_consensus.cc:3037] T 673f41d0ccba434b8b9ca6c3aa353ccb P 4fc803d2c9b34e5881f98d2ccc5b6d82 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:58.046207 15296 raft_consensus.cc:3037] T 673f41d0ccba434b8b9ca6c3aa353ccb P d079a46cbb004d438f1ddf3c4886090b [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:58.047565 15296 raft_consensus.cc:2454] T 673f41d0ccba434b8b9ca6c3aa353ccb P d079a46cbb004d438f1ddf3c4886090b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9deca17b4a7d46c5aadf4ecf88d21171 in term 1.
I20260812 11:13:58.047565 15713 raft_consensus.cc:2454] T 673f41d0ccba434b8b9ca6c3aa353ccb P 4fc803d2c9b34e5881f98d2ccc5b6d82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9deca17b4a7d46c5aadf4ecf88d21171 in term 1.
I20260812 11:13:58.047968 15402 leader_election.cc:302] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4fc803d2c9b34e5881f98d2ccc5b6d82, 9deca17b4a7d46c5aadf4ecf88d21171, d079a46cbb004d438f1ddf3c4886090b; no voters:
I20260812 11:13:58.048225 15829 raft_consensus.cc:2781] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:13:58.048486 15829 raft_consensus.cc:686] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171 [term 1 LEADER]: Becoming Leader. State: Replica: 9deca17b4a7d46c5aadf4ecf88d21171, State: Running, Role: LEADER
I20260812 11:13:58.048862 15829 consensus_queue.cc:237] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc803d2c9b34e5881f98d2ccc5b6d82" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46217 } } peers { permanent_uuid: "d079a46cbb004d438f1ddf3c4886090b" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 33237 } } peers { permanent_uuid: "9deca17b4a7d46c5aadf4ecf88d21171" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46709 } }
I20260812 11:13:58.052338 15072 catalog_manager.cc:5495] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9deca17b4a7d46c5aadf4ecf88d21171 (127.17.44.194). New cstate: current_term: 1 leader_uuid: "9deca17b4a7d46c5aadf4ecf88d21171" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc803d2c9b34e5881f98d2ccc5b6d82" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d079a46cbb004d438f1ddf3c4886090b" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 33237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9deca17b4a7d46c5aadf4ecf88d21171" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 46709 } health_report { overall_health: HEALTHY } } }
W20260812 11:13:58.055135 15577 tablet.cc:2369] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:13:58.066457 15369 tablet.cc:2369] T 673f41d0ccba434b8b9ca6c3aa353ccb P d079a46cbb004d438f1ddf3c4886090b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:13:58.085281 15819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:58.080448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57684 (local address 127.17.44.194:46709)
0812 11:13:58.080642 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:58.080646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:58.080680 (+ 34us) server_negotiation.cc:407] Connection header received
0812 11:13:58.080757 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:58.080759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:58.080817 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:58.080918 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:13:58.081268 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:58.081797 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:58.083202 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:58.083430 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:58.083519 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:58.084004 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:13:58.084659 (+ 655us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:13:58.084800 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:58.085053 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 11:13:58.085128 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260812 11:13:58.091965 15296 raft_consensus.cc:1261] T 673f41d0ccba434b8b9ca6c3aa353ccb P d079a46cbb004d438f1ddf3c4886090b [term 1 FOLLOWER]: Refusing update from remote peer 9deca17b4a7d46c5aadf4ecf88d21171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:13:58.092000 15713 raft_consensus.cc:1261] T 673f41d0ccba434b8b9ca6c3aa353ccb P 4fc803d2c9b34e5881f98d2ccc5b6d82 [term 1 FOLLOWER]: Refusing update from remote peer 9deca17b4a7d46c5aadf4ecf88d21171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:13:58.092974 15829 consensus_queue.cc:1035] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d079a46cbb004d438f1ddf3c4886090b" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 33237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:13:58.093204 15839 consensus_queue.cc:1035] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fc803d2c9b34e5881f98d2ccc5b6d82" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 46217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:13:58.101734 15843 mvcc.cc:205] Tried to move back new op lower bound from 7317640143215316992 to 7317640143050633216. Current Snapshot: MvccSnapshot[applied={T|T < 7317640143215316992}]
I20260812 11:13:58.101838 15845 mvcc.cc:205] Tried to move back new op lower bound from 7317640143215316992 to 7317640143050633216. Current Snapshot: MvccSnapshot[applied={T|T < 7317640143215316992}]
I20260812 11:13:58.114529 15072 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:13:58.114745 15072 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:13:58.117772 15072 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76f0c574443648fc99aeaab49bc66415: Sending DeleteTablet for 3 replicas of tablet 673f41d0ccba434b8b9ca6c3aa353ccb
I20260812 11:13:58.118706 15267 tablet_service.cc:1514] Processing DeleteTablet for tablet 673f41d0ccba434b8b9ca6c3aa353ccb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:13:58 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:58120
I20260812 11:13:58.118770 15684 tablet_service.cc:1514] Processing DeleteTablet for tablet 673f41d0ccba434b8b9ca6c3aa353ccb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:13:58 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:41260
I20260812 11:13:58.119333 15474 tablet_service.cc:1514] Processing DeleteTablet for tablet 673f41d0ccba434b8b9ca6c3aa353ccb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:13:58 UTC) from {username='kudu', principal='kudu/127.17.44.254@KRBTEST.COM'} at 127.0.0.1:57674
I20260812 11:13:58.119750 15855 tablet_replica.cc:330] T 673f41d0ccba434b8b9ca6c3aa353ccb P d079a46cbb004d438f1ddf3c4886090b: stopping tablet replica
I20260812 11:13:58.119817 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 15171
I20260812 11:13:58.130820 15858 tablet_replica.cc:330] T 673f41d0ccba434b8b9ca6c3aa353ccb P 4fc803d2c9b34e5881f98d2ccc5b6d82: stopping tablet replica
I20260812 11:13:58.131381 15858 raft_consensus.cc:2229] T 673f41d0ccba434b8b9ca6c3aa353ccb P 4fc803d2c9b34e5881f98d2ccc5b6d82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:13:58.132084 15858 raft_consensus.cc:2258] T 673f41d0ccba434b8b9ca6c3aa353ccb P 4fc803d2c9b34e5881f98d2ccc5b6d82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:13:58.132571 15860 tablet_replica.cc:330] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171: stopping tablet replica
I20260812 11:13:58.132853 15860 raft_consensus.cc:2229] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:13:58.134328 15858 ts_tablet_manager.cc:1905] T 673f41d0ccba434b8b9ca6c3aa353ccb P 4fc803d2c9b34e5881f98d2ccc5b6d82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:13:58.137204 15858 ts_tablet_manager.cc:1918] T 673f41d0ccba434b8b9ca6c3aa353ccb P 4fc803d2c9b34e5881f98d2ccc5b6d82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:13:58.137305 15858 log.cc:1196] T 673f41d0ccba434b8b9ca6c3aa353ccb P 4fc803d2c9b34e5881f98d2ccc5b6d82: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/673f41d0ccba434b8b9ca6c3aa353ccb
I20260812 11:13:58.137553 15860 raft_consensus.cc:2258] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:13:58.137639 15858 ts_tablet_manager.cc:1939] T 673f41d0ccba434b8b9ca6c3aa353ccb P 4fc803d2c9b34e5881f98d2ccc5b6d82: Deleting consensus metadata
I20260812 11:13:58.139108 15052 catalog_manager.cc:4841] TS 4fc803d2c9b34e5881f98d2ccc5b6d82 (127.17.44.195:46217): tablet 673f41d0ccba434b8b9ca6c3aa353ccb (table test-table [id=ba03124008e04898affe784041466e6e]) successfully deleted
I20260812 11:13:58.139395 15860 ts_tablet_manager.cc:1905] T 673f41d0ccba434b8b9ca6c3aa353ccb P 9deca17b4a7d46c5aadf4ecf88d21171: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 11:13:58.139974 15049 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.44.193:33237 (error 108)
I20260812 11:13:58.141039 15814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:58.140717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:33237 (local address 127.0.0.1:58130)
0812 11:13:58.140860 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:58.140937 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.193:33237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 11:13:58.141192 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 15373
W20260812 11:13:58.141300 15049 catalog_manager.cc:4584] TS d079a46cbb004d438f1ddf3c4886090b (127.17.44.193:33237): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 673f41d0ccba434b8b9ca6c3aa353ccb: Network error: Client connection negotiation failed: client connection to 127.17.44.193:33237: connect: Connection refused (error 111)
I20260812 11:13:58.157697 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 15583
I20260812 11:13:58.158401 15814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:58.158011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:46709 (local address 127.0.0.1:57700)
0812 11:13:58.158222 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:58.158283 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.194:46709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260812 11:13:58.176167 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 15017
2026-08-12T11:13:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2887 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T11:13:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:13:58Z Disabled control of system clock
I20260812 11:13:58.215521 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:46011
--webserver_interface=127.17.44.254
--webserver_port=0
--builtin_ntp_servers=127.17.44.212:46297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:46011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 11:13:58.334611 15866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:58.334862 15866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:58.334898 15866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:58.338832 15866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:13:58.338886 15866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:58.338902 15866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:58.338918 15866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:13:58.338958 15866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:13:58.343712 15866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:46297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:46011
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:46011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:58.344746 15866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:58.346072 15866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:58.354936 15874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:58.355034 15872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:58.354964 15871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:58.355249 15866 server_base.cc:1034] running on GCE node
I20260812 11:13:58.355765 15866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:58.356829 15866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:58.358173 15866 hybrid_clock.cc:648] HybridClock initialized: now 1786533238358155 us; error 54 us; skew 500 ppm
I20260812 11:13:58.360108 15866 webserver.cc:460] Webserver started at http://127.17.44.254:46095/ using document root <none> and password file <none>
I20260812 11:13:58.360646 15866 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:58.360703 15866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:58.360868 15866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:13:58.363025 15866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "30a9234002bf40a18bbbdaf3d92baa94"
format_stamp: "Formatted at 2026-08-12 11:13:58 on dist-test-slave-drl0"
server_key: "bba0c16fcc216158fffbff204f1fa4f1"
server_key_iv: "cc59a9a5ea7825cff7cc259301551e66"
server_key_version: "encryptionkey@0"
I20260812 11:13:58.363543 15866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "30a9234002bf40a18bbbdaf3d92baa94"
format_stamp: "Formatted at 2026-08-12 11:13:58 on dist-test-slave-drl0"
server_key: "bba0c16fcc216158fffbff204f1fa4f1"
server_key_iv: "cc59a9a5ea7825cff7cc259301551e66"
server_key_version: "encryptionkey@0"
I20260812 11:13:58.368448 15866 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 11:13:58.371188 15887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:58.372696 15866 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 11:13:58.372813 15866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "30a9234002bf40a18bbbdaf3d92baa94"
format_stamp: "Formatted at 2026-08-12 11:13:58 on dist-test-slave-drl0"
server_key: "bba0c16fcc216158fffbff204f1fa4f1"
server_key_iv: "cc59a9a5ea7825cff7cc259301551e66"
server_key_version: "encryptionkey@0"
I20260812 11:13:58.372961 15866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:58.391363 15866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:58.392277 15866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:58.392488 15866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:58.399905 15866 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:46011
I20260812 11:13:58.399917 15967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:46011 every 8 connection(s)
I20260812 11:13:58.400857 15866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:13:58.402138 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 15866
I20260812 11:13:58.402285 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:13:58.402694 17587 external_mini_cluster.cc:1428] Setting key 918aeb45e60b4b72d5d1d50a65358edb
I20260812 11:13:58.404829 15968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:58.411700 15968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30a9234002bf40a18bbbdaf3d92baa94: Bootstrap starting.
I20260812 11:13:58.414005 15973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:58.404608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43780 (local address 127.17.44.254:46011)
0812 11:13:58.405480 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:58.405502 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:58.405539 (+ 37us) server_negotiation.cc:407] Connection header received
0812 11:13:58.406213 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:58.406229 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:58.406604 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:58.406886 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:58.407473 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:58.408859 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:58.410395 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:58.410905 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:58.411122 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:58.412249 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:58.412254 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:58.412266 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:13:58.412649 (+ 383us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:58.412704 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:13:58.412733 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:58.412820 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:58.412999 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 11:13:58.413297 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":200,"threads_started":1}
I20260812 11:13:58.414163 15968 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30a9234002bf40a18bbbdaf3d92baa94: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:58.414901 15968 log.cc:824] T 00000000000000000000000000000000 P 30a9234002bf40a18bbbdaf3d92baa94: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:58.416790 15968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30a9234002bf40a18bbbdaf3d92baa94: No bootstrap required, opened a new log
I20260812 11:13:58.420260 15968 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30a9234002bf40a18bbbdaf3d92baa94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a9234002bf40a18bbbdaf3d92baa94" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 46011 } }
I20260812 11:13:58.420499 15968 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30a9234002bf40a18bbbdaf3d92baa94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:58.420581 15968 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30a9234002bf40a18bbbdaf3d92baa94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30a9234002bf40a18bbbdaf3d92baa94, State: Initialized, Role: FOLLOWER
I20260812 11:13:58.421042 15968 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30a9234002bf40a18bbbdaf3d92baa94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a9234002bf40a18bbbdaf3d92baa94" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 46011 } }
I20260812 11:13:58.421223 15968 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30a9234002bf40a18bbbdaf3d92baa94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:13:58.421316 15968 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30a9234002bf40a18bbbdaf3d92baa94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:13:58.421422 15968 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30a9234002bf40a18bbbdaf3d92baa94 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:58.422348 15968 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30a9234002bf40a18bbbdaf3d92baa94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a9234002bf40a18bbbdaf3d92baa94" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 46011 } }
I20260812 11:13:58.422688 15968 leader_election.cc:302] T 00000000000000000000000000000000 P 30a9234002bf40a18bbbdaf3d92baa94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30a9234002bf40a18bbbdaf3d92baa94; no voters:
I20260812 11:13:58.423039 15968 leader_election.cc:288] T 00000000000000000000000000000000 P 30a9234002bf40a18bbbdaf3d92baa94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:13:58.423152 15978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30a9234002bf40a18bbbdaf3d92baa94 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:13:58.423403 15978 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30a9234002bf40a18bbbdaf3d92baa94 [term 1 LEADER]: Becoming Leader. State: Replica: 30a9234002bf40a18bbbdaf3d92baa94, State: Running, Role: LEADER
I20260812 11:13:58.423707 15978 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30a9234002bf40a18bbbdaf3d92baa94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a9234002bf40a18bbbdaf3d92baa94" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 46011 } }
I20260812 11:13:58.424276 15968 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30a9234002bf40a18bbbdaf3d92baa94 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:13:58.425495 15984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30a9234002bf40a18bbbdaf3d92baa94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30a9234002bf40a18bbbdaf3d92baa94. Latest consensus state: current_term: 1 leader_uuid: "30a9234002bf40a18bbbdaf3d92baa94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a9234002bf40a18bbbdaf3d92baa94" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 46011 } } }
I20260812 11:13:58.425639 15984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30a9234002bf40a18bbbdaf3d92baa94 [sys.catalog]: This master's current role is: LEADER
I20260812 11:13:58.425506 15982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30a9234002bf40a18bbbdaf3d92baa94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30a9234002bf40a18bbbdaf3d92baa94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a9234002bf40a18bbbdaf3d92baa94" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 46011 } } }
I20260812 11:13:58.425948 15982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30a9234002bf40a18bbbdaf3d92baa94 [sys.catalog]: This master's current role is: LEADER
I20260812 11:13:58.426119 15994 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:13:58.429080 15994 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:13:58.436170 15994 catalog_manager.cc:1347] Generated new cluster ID: 37fb13068539476495118e6b8247e915
I20260812 11:13:58.436268 15994 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:13:58.470454 15994 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:13:58.471650 15994 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:13:58.481423 15994 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30a9234002bf40a18bbbdaf3d92baa94: Generated new TSK 0
I20260812 11:13:58.482146 15994 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:13:58.496243 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.193:0
--local_ip_for_outbound_sockets=127.17.44.193
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:46011
--builtin_ntp_servers=127.17.44.212:46297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 11:13:58.627008 16007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:58.627419 16007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:58.627498 16007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:58.636119 16007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:58.636281 16007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:58.636365 16007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.193
I20260812 11:13:58.641659 16007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:46297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.44.193
--webserver_port=0
--tserver_master_addrs=127.17.44.254:46011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.193
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:58.643504 16007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:58.646055 16007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:58.657186 16015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:58.657196 16020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:58.657231 16017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:58.657253 16007 server_base.cc:1034] running on GCE node
I20260812 11:13:58.657831 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:58.658484 16007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:58.659723 16007 hybrid_clock.cc:648] HybridClock initialized: now 1786533238659617 us; error 122 us; skew 500 ppm
I20260812 11:13:58.661593 16007 webserver.cc:460] Webserver started at http://127.17.44.193:45947/ using document root <none> and password file <none>
I20260812 11:13:58.662207 16007 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:58.662314 16007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:58.662551 16007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:13:58.664657 16007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-0/data/instance:
uuid: "f7fd495c26af4f529996e7a3bf939214"
format_stamp: "Formatted at 2026-08-12 11:13:58 on dist-test-slave-drl0"
server_key: "19559c5a6405fcfa4840f799fb16b148"
server_key_iv: "8ea51fc91dcf19384a6f819baca81654"
server_key_version: "encryptionkey@0"
I20260812 11:13:58.665264 16007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance:
uuid: "f7fd495c26af4f529996e7a3bf939214"
format_stamp: "Formatted at 2026-08-12 11:13:58 on dist-test-slave-drl0"
server_key: "19559c5a6405fcfa4840f799fb16b148"
server_key_iv: "8ea51fc91dcf19384a6f819baca81654"
server_key_version: "encryptionkey@0"
I20260812 11:13:58.669437 16007 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260812 11:13:58.672080 16029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:58.673404 16007 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:13:58.673556 16007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-0/wal
uuid: "f7fd495c26af4f529996e7a3bf939214"
format_stamp: "Formatted at 2026-08-12 11:13:58 on dist-test-slave-drl0"
server_key: "19559c5a6405fcfa4840f799fb16b148"
server_key_iv: "8ea51fc91dcf19384a6f819baca81654"
server_key_version: "encryptionkey@0"
I20260812 11:13:58.673673 16007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:58.693969 16007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:58.694681 16007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:58.694900 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:58.695567 16007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:13:58.696573 16007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:13:58.696648 16007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:58.696717 16007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:13:58.696766 16007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:58.707448 16007 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.193:44801
I20260812 11:13:58.707492 16234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.193:44801 every 8 connection(s)
I20260812 11:13:58.708429 16007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-0/data/info.pb
I20260812 11:13:58.716393 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 16007
I20260812 11:13:58.716605 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-0/wal/instance
I20260812 11:13:58.716984 17587 external_mini_cluster.cc:1428] Setting key 337fb6704e2fd6d0626addb3d13c9b62
I20260812 11:13:58.718997 15973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:58.710783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:41751 (local address 127.17.44.254:46011)
0812 11:13:58.711031 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:58.711035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:58.712280 (+ 1245us) server_negotiation.cc:407] Connection header received
0812 11:13:58.713737 (+ 1457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:58.713740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:58.713816 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:58.713934 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:58.714825 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:58.715427 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:58.716578 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:58.716885 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:58.716996 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:58.718164 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:58.718168 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:58.718227 (+ 59us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:13:58.718256 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:58.718286 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:13:58.718308 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:58.718462 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:58.718697 (+ 235us) server_negotiation.cc:299] Negotiation successful
0812 11:13:58.718795 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 11:13:58.719530 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.194:0
--local_ip_for_outbound_sockets=127.17.44.194
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:46011
--builtin_ntp_servers=127.17.44.212:46297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 11:13:58.720278 16239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:58.711338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:46011 (local address 127.17.44.193:41751)
0812 11:13:58.712068 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:58.712109 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:58.713514 (+ 1405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:58.714104 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:58.714111 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:58.714189 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:13:58.714667 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:58.714679 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:58.715560 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:58.715564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:58.716431 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:58.716441 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:58.717049 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:58.717054 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:58.717221 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:58.717890 (+ 669us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:13:58.717892 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:13:58.717930 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:13:58.717991 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:13:58.718475 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:13:58.718496 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:13:58.718507 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 11:13:58.718697 (+ 190us) client_negotiation.cc:240] Negotiation successful
0812 11:13:58.719173 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":136,"threads_started":1}
I20260812 11:13:58.722061 16236 heartbeater.cc:347] Connected to a master server at 127.17.44.254:46011
I20260812 11:13:58.722436 16236 heartbeater.cc:464] Registering TS with master...
I20260812 11:13:58.723407 16236 heartbeater.cc:511] Master 127.17.44.254:46011 requested a full tablet report, sending...
I20260812 11:13:58.725689 15912 ts_manager.cc:194] Registered new tserver with Master: f7fd495c26af4f529996e7a3bf939214 (127.17.44.193:44801)
I20260812 11:13:58.727511 15912 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.44.193:41751
W20260812 11:13:58.843470 16240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:58.843716 16240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:58.843755 16240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:58.849313 16240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:58.849431 16240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:58.849501 16240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.194
I20260812 11:13:58.855938 16240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:46297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.44.194
--webserver_port=0
--tserver_master_addrs=127.17.44.254:46011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.194
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:58.857028 16240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:58.858494 16240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:58.868561 16253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:58.868631 16250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:58.868719 16249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:58.868631 16240 server_base.cc:1034] running on GCE node
I20260812 11:13:58.869387 16240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:58.870277 16240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:58.871516 16240 hybrid_clock.cc:648] HybridClock initialized: now 1786533238871491 us; error 41 us; skew 500 ppm
I20260812 11:13:58.873557 16240 webserver.cc:460] Webserver started at http://127.17.44.194:36869/ using document root <none> and password file <none>
I20260812 11:13:58.874168 16240 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:58.874265 16240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:58.874516 16240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:13:58.876704 16240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-1/data/instance:
uuid: "4c051915ee414454a38e5584dda44ada"
format_stamp: "Formatted at 2026-08-12 11:13:58 on dist-test-slave-drl0"
server_key: "33f2daeff07e64b9b652e6c210a42e9a"
server_key_iv: "409b538492ec465bd645e14551129a34"
server_key_version: "encryptionkey@0"
I20260812 11:13:58.877501 16240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance:
uuid: "4c051915ee414454a38e5584dda44ada"
format_stamp: "Formatted at 2026-08-12 11:13:58 on dist-test-slave-drl0"
server_key: "33f2daeff07e64b9b652e6c210a42e9a"
server_key_iv: "409b538492ec465bd645e14551129a34"
server_key_version: "encryptionkey@0"
I20260812 11:13:58.881407 16240 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 11:13:58.884161 16271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:58.885645 16240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:13:58.885890 16240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-1/wal
uuid: "4c051915ee414454a38e5584dda44ada"
format_stamp: "Formatted at 2026-08-12 11:13:58 on dist-test-slave-drl0"
server_key: "33f2daeff07e64b9b652e6c210a42e9a"
server_key_iv: "409b538492ec465bd645e14551129a34"
server_key_version: "encryptionkey@0"
I20260812 11:13:58.886042 16240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:58.902511 16240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:58.903538 16240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:58.903777 16240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:58.904609 16240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:13:58.905789 16240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:13:58.905875 16240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:58.905951 16240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:13:58.905997 16240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:58.915594 16240 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.194:41537
I20260812 11:13:58.915665 16461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.194:41537 every 8 connection(s)
I20260812 11:13:58.916594 16240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-1/data/info.pb
I20260812 11:13:58.920881 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 16240
I20260812 11:13:58.921121 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-1/wal/instance
I20260812 11:13:58.921655 17587 external_mini_cluster.cc:1428] Setting key 19d8f0c5da544e939c78cce83a8e04b0
I20260812 11:13:58.923792 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.44.195:0
--local_ip_for_outbound_sockets=127.17.44.195
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:46011
--builtin_ntp_servers=127.17.44.212:46297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 11:13:58.927147 15973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:58.918655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.194:55919 (local address 127.17.44.254:46011)
0812 11:13:58.918811 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:58.918816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:58.919968 (+ 1152us) server_negotiation.cc:407] Connection header received
0812 11:13:58.921486 (+ 1518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:58.921491 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:58.921551 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:58.921660 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:58.923150 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:58.923684 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:58.925055 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:58.925282 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:58.925401 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:58.926609 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:58.926620 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:58.926623 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:13:58.926635 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:58.926653 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:13:58.926661 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:58.926740 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:58.926903 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 11:13:58.926998 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 11:13:58.928735 16464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:58.919122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:46011 (local address 127.17.44.194:55919)
0812 11:13:58.919796 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:58.919835 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:58.921213 (+ 1378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:58.922166 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:58.922174 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:58.922263 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:13:58.922982 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:58.923012 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:58.923822 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:58.923826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:58.924927 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:58.924938 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:58.925452 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:58.925457 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:58.925653 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:58.926328 (+ 675us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:13:58.926330 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:13:58.926376 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:13:58.926441 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:13:58.926744 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:13:58.926756 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:13:58.926763 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 11:13:58.926902 (+ 139us) client_negotiation.cc:240] Negotiation successful
0812 11:13:58.927195 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":115,"threads_started":1}
I20260812 11:13:58.932317 16462 heartbeater.cc:347] Connected to a master server at 127.17.44.254:46011
I20260812 11:13:58.932633 16462 heartbeater.cc:464] Registering TS with master...
I20260812 11:13:58.933399 16462 heartbeater.cc:511] Master 127.17.44.254:46011 requested a full tablet report, sending...
I20260812 11:13:58.934979 15912 ts_manager.cc:194] Registered new tserver with Master: 4c051915ee414454a38e5584dda44ada (127.17.44.194:41537)
I20260812 11:13:58.936197 15912 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.44.194:55919
W20260812 11:13:59.050236 16465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:13:59.050619 16465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:13:59.050720 16465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:13:59.054916 16465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:13:59.054994 16465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:13:59.055095 16465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.44.195
I20260812 11:13:59.059955 16465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:46297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.44.195
--webserver_port=0
--tserver_master_addrs=127.17.44.254:46011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.44.195
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:13:59.061081 16465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:13:59.062532 16465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:13:59.073796 16474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:59.073835 16476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:13:59.073817 16473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:13:59.073904 16465 server_base.cc:1034] running on GCE node
I20260812 11:13:59.074541 16465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:13:59.075287 16465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:13:59.076573 16465 hybrid_clock.cc:648] HybridClock initialized: now 1786533239076527 us; error 62 us; skew 500 ppm
I20260812 11:13:59.078712 16465 webserver.cc:460] Webserver started at http://127.17.44.195:39235/ using document root <none> and password file <none>
I20260812 11:13:59.079407 16465 fs_manager.cc:359] Metadata directory not provided
I20260812 11:13:59.079504 16465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:13:59.079751 16465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:13:59.081844 16465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-2/data/instance:
uuid: "a253fe3bf2f1416c865caad242d6a513"
format_stamp: "Formatted at 2026-08-12 11:13:59 on dist-test-slave-drl0"
server_key: "72a0bea19433c824432209b877c13c78"
server_key_iv: "4788d9e13adafb602965305a7a1119fd"
server_key_version: "encryptionkey@0"
I20260812 11:13:59.082459 16465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance:
uuid: "a253fe3bf2f1416c865caad242d6a513"
format_stamp: "Formatted at 2026-08-12 11:13:59 on dist-test-slave-drl0"
server_key: "72a0bea19433c824432209b877c13c78"
server_key_iv: "4788d9e13adafb602965305a7a1119fd"
server_key_version: "encryptionkey@0"
I20260812 11:13:59.086664 16465 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 11:13:59.089918 16486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:59.091645 16465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:13:59.091799 16465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-2/wal
uuid: "a253fe3bf2f1416c865caad242d6a513"
format_stamp: "Formatted at 2026-08-12 11:13:59 on dist-test-slave-drl0"
server_key: "72a0bea19433c824432209b877c13c78"
server_key_iv: "4788d9e13adafb602965305a7a1119fd"
server_key_version: "encryptionkey@0"
I20260812 11:13:59.091922 16465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:13:59.116142 16465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:13:59.116899 16465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:13:59.117110 16465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:13:59.117741 16465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 11:13:59.118703 16465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 11:13:59.118796 16465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 11:13:59.118873 16465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 11:13:59.118947 16465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260812 11:13:59.129940 16465 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.195:39643
I20260812 11:13:59.129982 16673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.195:39643 every 8 connection(s)
I20260812 11:13:59.130863 16465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-2/data/info.pb
I20260812 11:13:59.133240 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 16465
I20260812 11:13:59.133381 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-2/wal/instance
I20260812 11:13:59.133770 17587 external_mini_cluster.cc:1428] Setting key 588a948bbe19e20e690823925deb1652
I20260812 11:13:59.140493 15973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:59.132845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.195:34709 (local address 127.17.44.254:46011)
0812 11:13:59.133070 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:59.133076 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:59.134270 (+ 1194us) server_negotiation.cc:407] Connection header received
0812 11:13:59.135855 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:59.135858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:59.135916 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:59.136019 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:59.136899 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.137433 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.138554 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.138775 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.138885 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:59.140005 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:59.140008 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:59.140010 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:13:59.140022 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:59.140039 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:13:59.140047 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:59.140129 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:59.140292 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 11:13:59.140354 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 11:13:59.141397 16676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:59.133198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:46011 (local address 127.17.44.195:34709)
0812 11:13:59.134068 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:59.134123 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:59.135634 (+ 1511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:59.136199 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:59.136206 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:59.136287 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:13:59.136760 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:59.136772 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.137563 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.137567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:59.138420 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:59.138429 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.138949 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.138954 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:59.139123 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:59.139717 (+ 594us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 11:13:59.139719 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:13:59.139765 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 11:13:59.139825 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:13:59.140131 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:13:59.140143 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:13:59.140151 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 11:13:59.140291 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 11:13:59.140563 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":179,"threads_started":1}
I20260812 11:13:59.142562 16674 heartbeater.cc:347] Connected to a master server at 127.17.44.254:46011
I20260812 11:13:59.142804 16674 heartbeater.cc:464] Registering TS with master...
I20260812 11:13:59.143330 16674 heartbeater.cc:511] Master 127.17.44.254:46011 requested a full tablet report, sending...
I20260812 11:13:59.144660 15912 ts_manager.cc:194] Registered new tserver with Master: a253fe3bf2f1416c865caad242d6a513 (127.17.44.195:39643)
I20260812 11:13:59.145198 15912 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.44.195:34709
I20260812 11:13:59.148162 17587 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 11:13:59.154340 15973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:59.150602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43782 (local address 127.17.44.254:46011)
0812 11:13:59.150718 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:59.150722 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:59.150823 (+ 101us) server_negotiation.cc:407] Connection header received
0812 11:13:59.150916 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:59.150918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:59.151002 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:59.151082 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:13:59.151376 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.151878 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.152801 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.153012 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.153097 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:59.153893 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:13:59.153896 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:13:59.153898 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:13:59.153909 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:13:59.153926 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:13:59.153934 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:13:59.153999 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:59.154191 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 11:13:59.154239 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 11:13:59.157469 15912 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 11:13:59.160787 15912 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 11:13:59.177740 16707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:59.172162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39038 (local address 127.17.44.195:39643)
0812 11:13:59.173046 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:59.173059 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:59.173085 (+ 26us) server_negotiation.cc:407] Connection header received
0812 11:13:59.173176 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:59.173180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:59.173337 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:59.173512 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:59.173804 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.175135 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.176344 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.177027 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.177156 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:59.177353 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:59.177465 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 11:13:59.177559 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":771,"thread_start_us":182,"threads_started":1}
I20260812 11:13:59.177733 16703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:59.171636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:39643 (local address 127.0.0.1:39038)
0812 11:13:59.172659 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:59.172677 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:59.172782 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:59.173504 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:59.173510 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:59.173533 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:59.173674 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:59.173681 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.175267 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.175271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:59.176214 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:59.176225 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.177156 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.177160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:59.177264 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:59.177299 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 11:13:59.177435 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 11:13:59.177558 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":946,"mutex_wait_us":28,"thread_start_us":121,"threads_started":1}
I20260812 11:13:59.178262 16705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:59.171289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:41537 (local address 127.0.0.1:45056)
0812 11:13:59.172343 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:59.172409 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:59.172590 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:59.173674 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:59.173677 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:59.173693 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:59.173802 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:59.173808 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.174850 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.174855 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:59.176183 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:59.176193 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.177365 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.177368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:59.177479 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:59.177504 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 11:13:59.177562 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 11:13:59.177629 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"mutex_wait_us":1,"thread_start_us":65,"threads_started":1}
I20260812 11:13:59.178632 16706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:59.171595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45056 (local address 127.17.44.194:41537)
0812 11:13:59.172929 (+ 1334us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:59.172945 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:59.172976 (+ 31us) server_negotiation.cc:407] Connection header received
0812 11:13:59.173212 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:59.173219 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:59.173431 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:59.173603 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:59.173924 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.174732 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.176308 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.177207 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.177787 (+ 580us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:59.178109 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:59.178157 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 11:13:59.178467 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1206,"thread_start_us":237,"threads_started":1}
I20260812 11:13:59.179265 16709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:59.172453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:44801 (local address 127.0.0.1:54180)
0812 11:13:59.172962 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:59.172976 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:59.173057 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:59.173935 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:59.173938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:59.173956 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:59.174043 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:59.174049 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.175378 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.175381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:59.177364 (+ 1983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:59.177372 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.178874 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.178878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:59.178999 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:59.179025 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 11:13:59.179089 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 11:13:59.179151 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":104,"threads_started":1}
I20260812 11:13:59.179410 16710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:59.172719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54180 (local address 127.17.44.193:44801)
0812 11:13:59.173347 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:59.173353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:59.173377 (+ 24us) server_negotiation.cc:407] Connection header received
0812 11:13:59.173453 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:59.173458 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:59.173619 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:59.173758 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:59.174192 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.175265 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.177493 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.178762 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.178870 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:59.179102 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:59.179150 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 11:13:59.179233 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":100,"threads_started":1}
I20260812 11:13:59.180881 16574 tablet_service.cc:1467] Processing CreateTablet for tablet f298ec8bc504452b939c7980161b1b68 (DEFAULT_TABLE table=test-table [id=fc64e1650e5d42d3b96db3156b79f8ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:13:59.181839 16574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f298ec8bc504452b939c7980161b1b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:59.181792 16351 tablet_service.cc:1467] Processing CreateTablet for tablet f298ec8bc504452b939c7980161b1b68 (DEFAULT_TABLE table=test-table [id=fc64e1650e5d42d3b96db3156b79f8ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:13:59.181792 16111 tablet_service.cc:1467] Processing CreateTablet for tablet f298ec8bc504452b939c7980161b1b68 (DEFAULT_TABLE table=test-table [id=fc64e1650e5d42d3b96db3156b79f8ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 11:13:59.182701 16111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f298ec8bc504452b939c7980161b1b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:59.182701 16351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f298ec8bc504452b939c7980161b1b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:13:59.188192 16711 tablet_bootstrap.cc:492] T f298ec8bc504452b939c7980161b1b68 P a253fe3bf2f1416c865caad242d6a513: Bootstrap starting.
I20260812 11:13:59.190245 16713 tablet_bootstrap.cc:492] T f298ec8bc504452b939c7980161b1b68 P 4c051915ee414454a38e5584dda44ada: Bootstrap starting.
I20260812 11:13:59.190641 16711 tablet_bootstrap.cc:654] T f298ec8bc504452b939c7980161b1b68 P a253fe3bf2f1416c865caad242d6a513: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:59.191499 16714 tablet_bootstrap.cc:492] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214: Bootstrap starting.
I20260812 11:13:59.191635 16711 log.cc:824] T f298ec8bc504452b939c7980161b1b68 P a253fe3bf2f1416c865caad242d6a513: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:59.192461 16713 tablet_bootstrap.cc:654] T f298ec8bc504452b939c7980161b1b68 P 4c051915ee414454a38e5584dda44ada: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:59.193240 16713 log.cc:824] T f298ec8bc504452b939c7980161b1b68 P 4c051915ee414454a38e5584dda44ada: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:59.193924 16714 tablet_bootstrap.cc:654] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214: Neither blocks nor log segments found. Creating new log.
I20260812 11:13:59.194375 16711 tablet_bootstrap.cc:492] T f298ec8bc504452b939c7980161b1b68 P a253fe3bf2f1416c865caad242d6a513: No bootstrap required, opened a new log
I20260812 11:13:59.194576 16711 ts_tablet_manager.cc:1397] T f298ec8bc504452b939c7980161b1b68 P a253fe3bf2f1416c865caad242d6a513: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260812 11:13:59.194749 16714 log.cc:824] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214: Log is configured to *not* fsync() on all Append() calls
I20260812 11:13:59.194916 16713 tablet_bootstrap.cc:492] T f298ec8bc504452b939c7980161b1b68 P 4c051915ee414454a38e5584dda44ada: No bootstrap required, opened a new log
I20260812 11:13:59.195107 16713 ts_tablet_manager.cc:1397] T f298ec8bc504452b939c7980161b1b68 P 4c051915ee414454a38e5584dda44ada: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260812 11:13:59.196739 16714 tablet_bootstrap.cc:492] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214: No bootstrap required, opened a new log
I20260812 11:13:59.196923 16714 ts_tablet_manager.cc:1397] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 11:13:59.198541 16711 raft_consensus.cc:348] T f298ec8bc504452b939c7980161b1b68 P a253fe3bf2f1416c865caad242d6a513 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c051915ee414454a38e5584dda44ada" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41537 } } peers { permanent_uuid: "a253fe3bf2f1416c865caad242d6a513" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 39643 } } peers { permanent_uuid: "f7fd495c26af4f529996e7a3bf939214" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44801 } }
I20260812 11:13:59.198779 16711 raft_consensus.cc:374] T f298ec8bc504452b939c7980161b1b68 P a253fe3bf2f1416c865caad242d6a513 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:59.198832 16711 raft_consensus.cc:729] T f298ec8bc504452b939c7980161b1b68 P a253fe3bf2f1416c865caad242d6a513 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a253fe3bf2f1416c865caad242d6a513, State: Initialized, Role: FOLLOWER
I20260812 11:13:59.199306 16711 consensus_queue.cc:260] T f298ec8bc504452b939c7980161b1b68 P a253fe3bf2f1416c865caad242d6a513 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c051915ee414454a38e5584dda44ada" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41537 } } peers { permanent_uuid: "a253fe3bf2f1416c865caad242d6a513" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 39643 } } peers { permanent_uuid: "f7fd495c26af4f529996e7a3bf939214" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44801 } }
I20260812 11:13:59.199411 16713 raft_consensus.cc:348] T f298ec8bc504452b939c7980161b1b68 P 4c051915ee414454a38e5584dda44ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c051915ee414454a38e5584dda44ada" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41537 } } peers { permanent_uuid: "a253fe3bf2f1416c865caad242d6a513" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 39643 } } peers { permanent_uuid: "f7fd495c26af4f529996e7a3bf939214" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44801 } }
I20260812 11:13:59.199656 16713 raft_consensus.cc:374] T f298ec8bc504452b939c7980161b1b68 P 4c051915ee414454a38e5584dda44ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:59.199714 16713 raft_consensus.cc:729] T f298ec8bc504452b939c7980161b1b68 P 4c051915ee414454a38e5584dda44ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c051915ee414454a38e5584dda44ada, State: Initialized, Role: FOLLOWER
I20260812 11:13:59.200085 16674 heartbeater.cc:503] Master 127.17.44.254:46011 was elected leader, sending a full tablet report...
I20260812 11:13:59.200151 16713 consensus_queue.cc:260] T f298ec8bc504452b939c7980161b1b68 P 4c051915ee414454a38e5584dda44ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c051915ee414454a38e5584dda44ada" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41537 } } peers { permanent_uuid: "a253fe3bf2f1416c865caad242d6a513" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 39643 } } peers { permanent_uuid: "f7fd495c26af4f529996e7a3bf939214" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44801 } }
I20260812 11:13:59.200325 16711 ts_tablet_manager.cc:1428] T f298ec8bc504452b939c7980161b1b68 P a253fe3bf2f1416c865caad242d6a513: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260812 11:13:59.201084 16713 ts_tablet_manager.cc:1428] T f298ec8bc504452b939c7980161b1b68 P 4c051915ee414454a38e5584dda44ada: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 11:13:59.201086 16714 raft_consensus.cc:348] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c051915ee414454a38e5584dda44ada" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41537 } } peers { permanent_uuid: "a253fe3bf2f1416c865caad242d6a513" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 39643 } } peers { permanent_uuid: "f7fd495c26af4f529996e7a3bf939214" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44801 } }
I20260812 11:13:59.201356 16714 raft_consensus.cc:374] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:13:59.201417 16714 raft_consensus.cc:729] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7fd495c26af4f529996e7a3bf939214, State: Initialized, Role: FOLLOWER
I20260812 11:13:59.201413 16462 heartbeater.cc:503] Master 127.17.44.254:46011 was elected leader, sending a full tablet report...
I20260812 11:13:59.201802 16714 consensus_queue.cc:260] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c051915ee414454a38e5584dda44ada" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41537 } } peers { permanent_uuid: "a253fe3bf2f1416c865caad242d6a513" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 39643 } } peers { permanent_uuid: "f7fd495c26af4f529996e7a3bf939214" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44801 } }
I20260812 11:13:59.202378 16236 heartbeater.cc:503] Master 127.17.44.254:46011 was elected leader, sending a full tablet report...
I20260812 11:13:59.202617 16714 ts_tablet_manager.cc:1428] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260812 11:13:59.210642 16237 tablet.cc:2369] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:13:59.355448 16722 raft_consensus.cc:482] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 11:13:59.355718 16722 raft_consensus.cc:504] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c051915ee414454a38e5584dda44ada" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41537 } } peers { permanent_uuid: "a253fe3bf2f1416c865caad242d6a513" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 39643 } } peers { permanent_uuid: "f7fd495c26af4f529996e7a3bf939214" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44801 } }
I20260812 11:13:59.356943 16722 leader_election.cc:288] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c051915ee414454a38e5584dda44ada (127.17.44.194:41537), a253fe3bf2f1416c865caad242d6a513 (127.17.44.195:39643)
I20260812 11:13:59.363348 16724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:59.357301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.194:41537 (local address 127.17.44.193:35917)
0812 11:13:59.357769 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:59.357782 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:59.357876 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:59.358211 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:59.358215 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:59.358264 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:59.358548 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:59.358557 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.359721 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.359725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:59.361392 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:59.361404 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.362587 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.362592 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:59.362834 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:59.362892 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 11:13:59.362999 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 11:13:59.363141 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":438,"spinlock_wait_cycles":10624,"thread_start_us":134,"threads_started":1}
I20260812 11:13:59.363356 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:59.357301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.195:39643 (local address 127.17.44.193:48687)
0812 11:13:59.357636 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:59.357654 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 11:13:59.357811 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:13:59.358211 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:13:59.358215 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:13:59.358263 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 11:13:59.358554 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:59.358559 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.359721 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.359725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:59.361393 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:13:59.361404 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.362587 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.362592 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:13:59.362826 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:59.362887 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 11:13:59.362999 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 11:13:59.363139 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":397,"spinlock_wait_cycles":31232,"thread_start_us":83,"threads_started":1}
I20260812 11:13:59.363481 16706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:59.357430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:35917 (local address 127.17.44.194:41537)
0812 11:13:59.357670 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:59.357673 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:59.357905 (+ 232us) server_negotiation.cc:407] Connection header received
0812 11:13:59.357969 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:59.357971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:59.358019 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:59.358101 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:59.358724 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.359571 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.361659 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.362277 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.362414 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:59.362472 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:59.363129 (+ 657us) server_negotiation.cc:299] Negotiation successful
0812 11:13:59.363280 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260812 11:13:59.363464 16707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:59.357460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.44.193:48687 (local address 127.17.44.195:39643)
0812 11:13:59.357605 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:59.357610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:59.357790 (+ 180us) server_negotiation.cc:407] Connection header received
0812 11:13:59.357946 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:59.357949 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:59.358006 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:59.358101 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 11:13:59.358724 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.359572 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.361653 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.362277 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.362414 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:59.362472 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:59.363129 (+ 657us) server_negotiation.cc:299] Negotiation successful
0812 11:13:59.363278 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 11:13:59.364264 16607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f298ec8bc504452b939c7980161b1b68" candidate_uuid: "f7fd495c26af4f529996e7a3bf939214" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a253fe3bf2f1416c865caad242d6a513" is_pre_election: true
I20260812 11:13:59.364629 16607 raft_consensus.cc:2454] T f298ec8bc504452b939c7980161b1b68 P a253fe3bf2f1416c865caad242d6a513 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7fd495c26af4f529996e7a3bf939214 in term 0.
I20260812 11:13:59.364673 16388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f298ec8bc504452b939c7980161b1b68" candidate_uuid: "f7fd495c26af4f529996e7a3bf939214" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c051915ee414454a38e5584dda44ada" is_pre_election: true
I20260812 11:13:59.365087 16388 raft_consensus.cc:2454] T f298ec8bc504452b939c7980161b1b68 P 4c051915ee414454a38e5584dda44ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7fd495c26af4f529996e7a3bf939214 in term 0.
I20260812 11:13:59.365163 16030 leader_election.cc:302] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a253fe3bf2f1416c865caad242d6a513, f7fd495c26af4f529996e7a3bf939214; no voters:
I20260812 11:13:59.365525 16722 raft_consensus.cc:2781] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 11:13:59.365700 16722 raft_consensus.cc:482] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 11:13:59.365759 16722 raft_consensus.cc:3037] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:59.367308 16722 raft_consensus.cc:504] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c051915ee414454a38e5584dda44ada" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41537 } } peers { permanent_uuid: "a253fe3bf2f1416c865caad242d6a513" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 39643 } } peers { permanent_uuid: "f7fd495c26af4f529996e7a3bf939214" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44801 } }
I20260812 11:13:59.367686 16722 leader_election.cc:288] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214 [CANDIDATE]: Term 1 election: Requested vote from peers 4c051915ee414454a38e5584dda44ada (127.17.44.194:41537), a253fe3bf2f1416c865caad242d6a513 (127.17.44.195:39643)
I20260812 11:13:59.368148 16388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f298ec8bc504452b939c7980161b1b68" candidate_uuid: "f7fd495c26af4f529996e7a3bf939214" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c051915ee414454a38e5584dda44ada"
I20260812 11:13:59.368212 16607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f298ec8bc504452b939c7980161b1b68" candidate_uuid: "f7fd495c26af4f529996e7a3bf939214" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a253fe3bf2f1416c865caad242d6a513"
I20260812 11:13:59.368332 16607 raft_consensus.cc:3037] T f298ec8bc504452b939c7980161b1b68 P a253fe3bf2f1416c865caad242d6a513 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:59.368332 16388 raft_consensus.cc:3037] T f298ec8bc504452b939c7980161b1b68 P 4c051915ee414454a38e5584dda44ada [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:13:59.369664 16607 raft_consensus.cc:2454] T f298ec8bc504452b939c7980161b1b68 P a253fe3bf2f1416c865caad242d6a513 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7fd495c26af4f529996e7a3bf939214 in term 1.
I20260812 11:13:59.369664 16388 raft_consensus.cc:2454] T f298ec8bc504452b939c7980161b1b68 P 4c051915ee414454a38e5584dda44ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7fd495c26af4f529996e7a3bf939214 in term 1.
I20260812 11:13:59.370031 16032 leader_election.cc:302] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c051915ee414454a38e5584dda44ada, f7fd495c26af4f529996e7a3bf939214; no voters:
I20260812 11:13:59.370247 16722 raft_consensus.cc:2781] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:13:59.370503 16722 raft_consensus.cc:686] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214 [term 1 LEADER]: Becoming Leader. State: Replica: f7fd495c26af4f529996e7a3bf939214, State: Running, Role: LEADER
I20260812 11:13:59.370896 16722 consensus_queue.cc:237] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c051915ee414454a38e5584dda44ada" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41537 } } peers { permanent_uuid: "a253fe3bf2f1416c865caad242d6a513" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 39643 } } peers { permanent_uuid: "f7fd495c26af4f529996e7a3bf939214" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44801 } }
I20260812 11:13:59.374662 15912 catalog_manager.cc:5495] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214 reported cstate change: term changed from 0 to 1, leader changed from <none> to f7fd495c26af4f529996e7a3bf939214 (127.17.44.193). New cstate: current_term: 1 leader_uuid: "f7fd495c26af4f529996e7a3bf939214" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c051915ee414454a38e5584dda44ada" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a253fe3bf2f1416c865caad242d6a513" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 39643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7fd495c26af4f529996e7a3bf939214" member_type: VOTER last_known_addr { host: "127.17.44.193" port: 44801 } health_report { overall_health: HEALTHY } } }
W20260812 11:13:59.381855 16675 tablet.cc:2369] T f298ec8bc504452b939c7980161b1b68 P a253fe3bf2f1416c865caad242d6a513: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 11:13:59.417987 16463 tablet.cc:2369] T f298ec8bc504452b939c7980161b1b68 P 4c051915ee414454a38e5584dda44ada: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 11:13:59.421051 16710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:59.417251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54194 (local address 127.17.44.193:44801)
0812 11:13:59.417384 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 11:13:59.417389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:13:59.417497 (+ 108us) server_negotiation.cc:407] Connection header received
0812 11:13:59.417596 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:13:59.417598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:13:59.417659 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:13:59.417761 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:13:59.418076 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.418643 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.419578 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:13:59.419812 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:13:59.419904 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:13:59.420198 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:13:59.420615 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:13:59.420708 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 11:13:59.420880 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 11:13:59.420933 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 11:13:59.427994 16388 raft_consensus.cc:1261] T f298ec8bc504452b939c7980161b1b68 P 4c051915ee414454a38e5584dda44ada [term 1 FOLLOWER]: Refusing update from remote peer f7fd495c26af4f529996e7a3bf939214: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:13:59.427994 16607 raft_consensus.cc:1261] T f298ec8bc504452b939c7980161b1b68 P a253fe3bf2f1416c865caad242d6a513 [term 1 FOLLOWER]: Refusing update from remote peer f7fd495c26af4f529996e7a3bf939214: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:13:59.428723 16722 consensus_queue.cc:1035] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c051915ee414454a38e5584dda44ada" member_type: VOTER last_known_addr { host: "127.17.44.194" port: 41537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:13:59.428997 16725 consensus_queue.cc:1035] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a253fe3bf2f1416c865caad242d6a513" member_type: VOTER last_known_addr { host: "127.17.44.195" port: 39643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:13:59.439746 16736 mvcc.cc:205] Tried to move back new op lower bound from 7317640148688863232 to 7317640148466626560. Current Snapshot: MvccSnapshot[applied={T|T < 7317640148688863232}]
I20260812 11:13:59.445686 15911 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:13:59.445889 15911 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 11:13:59.448587 15911 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 30a9234002bf40a18bbbdaf3d92baa94: Sending DeleteTablet for 3 replicas of tablet f298ec8bc504452b939c7980161b1b68
I20260812 11:13:59.449460 16574 tablet_service.cc:1514] Processing DeleteTablet for tablet f298ec8bc504452b939c7980161b1b68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:13:59 UTC) from {username='slave'} at 127.0.0.1:39038
I20260812 11:13:59.449517 16111 tablet_service.cc:1514] Processing DeleteTablet for tablet f298ec8bc504452b939c7980161b1b68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:13:59 UTC) from {username='slave'} at 127.0.0.1:54180
I20260812 11:13:59.449679 16351 tablet_service.cc:1514] Processing DeleteTablet for tablet f298ec8bc504452b939c7980161b1b68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 11:13:59 UTC) from {username='slave'} at 127.0.0.1:45056
I20260812 11:13:59.450026 16751 tablet_replica.cc:330] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214: stopping tablet replica
I20260812 11:13:59.450125 16750 tablet_replica.cc:330] T f298ec8bc504452b939c7980161b1b68 P a253fe3bf2f1416c865caad242d6a513: stopping tablet replica
I20260812 11:13:59.450454 16751 raft_consensus.cc:2229] T f298ec8bc504452b939c7980161b1b68 P f7fd495c26af4f529996e7a3bf939214 [term 1 LEADER]: Raft consensus shutting down.
I20260812 11:13:59.450507 16750 raft_consensus.cc:2229] T f298ec8bc504452b939c7980161b1b68 P a253fe3bf2f1416c865caad242d6a513 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:13:59.450695 16753 tablet_replica.cc:330] T f298ec8bc504452b939c7980161b1b68 P 4c051915ee414454a38e5584dda44ada: stopping tablet replica
I20260812 11:13:59.450784 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 16007
I20260812 11:13:59.451073 16750 raft_consensus.cc:2258] T f298ec8bc504452b939c7980161b1b68 P a253fe3bf2f1416c865caad242d6a513 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:13:59.454890 16750 ts_tablet_manager.cc:1905] T f298ec8bc504452b939c7980161b1b68 P a253fe3bf2f1416c865caad242d6a513: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:13:59.451112 16753 raft_consensus.cc:2229] T f298ec8bc504452b939c7980161b1b68 P 4c051915ee414454a38e5584dda44ada [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 11:13:59.456416 16753 raft_consensus.cc:2258] T f298ec8bc504452b939c7980161b1b68 P 4c051915ee414454a38e5584dda44ada [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 11:13:59.459415 16750 ts_tablet_manager.cc:1918] T f298ec8bc504452b939c7980161b1b68 P a253fe3bf2f1416c865caad242d6a513: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:13:59.459560 16750 log.cc:1196] T f298ec8bc504452b939c7980161b1b68 P a253fe3bf2f1416c865caad242d6a513: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-2/wal/wals/f298ec8bc504452b939c7980161b1b68
I20260812 11:13:59.460033 16750 ts_tablet_manager.cc:1939] T f298ec8bc504452b939c7980161b1b68 P a253fe3bf2f1416c865caad242d6a513: Deleting consensus metadata
I20260812 11:13:59.461125 16753 ts_tablet_manager.cc:1905] T f298ec8bc504452b939c7980161b1b68 P 4c051915ee414454a38e5584dda44ada: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 11:13:59.461287 15888 catalog_manager.cc:4841] TS a253fe3bf2f1416c865caad242d6a513 (127.17.44.195:39643): tablet f298ec8bc504452b939c7980161b1b68 (table test-table [id=fc64e1650e5d42d3b96db3156b79f8ca]) successfully deleted
I20260812 11:13:59.466022 16753 ts_tablet_manager.cc:1918] T f298ec8bc504452b939c7980161b1b68 P 4c051915ee414454a38e5584dda44ada: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 11:13:59.466336 16753 log.cc:1196] T f298ec8bc504452b939c7980161b1b68 P 4c051915ee414454a38e5584dda44ada: Deleting WAL directory at /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786532961723938-17587-0/minicluster-data/ts-1/wal/wals/f298ec8bc504452b939c7980161b1b68
I20260812 11:13:59.466988 16753 ts_tablet_manager.cc:1939] T f298ec8bc504452b939c7980161b1b68 P 4c051915ee414454a38e5584dda44ada: Deleting consensus metadata
I20260812 11:13:59.468716 15889 catalog_manager.cc:4841] TS 4c051915ee414454a38e5584dda44ada (127.17.44.194:41537): tablet f298ec8bc504452b939c7980161b1b68 (table test-table [id=fc64e1650e5d42d3b96db3156b79f8ca]) successfully deleted
W20260812 11:13:59.470043 15889 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.44.193:44801 (error 108)
I20260812 11:13:59.471434 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 16240
I20260812 11:13:59.471652 16709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:13:59.471146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.193:44801 (local address 127.0.0.1:54206)
0812 11:13:59.471411 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 11:13:59.471538 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.193:44801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260812 11:13:59.479890 15889 catalog_manager.cc:4584] TS f7fd495c26af4f529996e7a3bf939214 (127.17.44.193:44801): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f298ec8bc504452b939c7980161b1b68: Network error: Client connection negotiation failed: client connection to 127.17.44.193:44801: connect: Connection refused (error 111)
I20260812 11:13:59.488857 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 16465
I20260812 11:13:59.501456 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 15866
2026-08-12T11:13:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1324 ms)
[----------] 6 tests from EncryptionPolicyTest (12250 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:13:59 dist-test-slave-drl0 krb5kdc[16761](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:13:59 dist-test-slave-drl0 krb5kdc[16761](info): set up 2 sockets
Aug 12 11:13:59 dist-test-slave-drl0 krb5kdc[16761](info): commencing operation
krb5kdc: starting...
W20260812 11:14:00.801098 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:14:00 dist-test-slave-drl0 krb5kdc[16761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:14:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:14:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 11:14:00.938699 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37201
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:14:01.062203 16779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:14:01.062474 16779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:14:01.062517 16779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:14:01.066864 16779 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:14:01.066951 16779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:14:01.066973 16779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:14:01.066990 16779 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:14:01.067006 16779 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:14:01.072271 16779 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37201
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:14:01.073693 16779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:14:01.074985 16779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:14:01.083702 16787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:01.083707 16788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:01.083914 16790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:01.084290 16779 server_base.cc:1034] running on GCE node
I20260812 11:14:01.084833 16779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:14:01.085891 16779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:14:01.087034 16779 hybrid_clock.cc:648] HybridClock initialized: now 1786533241087000 us; error 47 us; skew 500 ppm
Aug 12 11:14:01 dist-test-slave-drl0 krb5kdc[16761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533241, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:14:01.129742 16779 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 11:14:01.130874 16779 webserver.cc:460] Webserver started at http://127.0.0.1:46661/ using document root <none> and password file <none>
I20260812 11:14:01.131587 16779 fs_manager.cc:359] Metadata directory not provided
I20260812 11:14:01.131655 16779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:14:01.131891 16779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:14:01.135012 16779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "41abea7da6354051a18b687f3f3d4839"
format_stamp: "Formatted at 2026-08-12 11:14:01 on dist-test-slave-drl0"
server_key: "02a0a917ec2024b1ef17dd4a76a564da"
server_key_iv: "f5d5a427fa4430f2edf106a1ea8f7246"
server_key_version: "encryptionkey@0"
I20260812 11:14:01.135931 16779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "41abea7da6354051a18b687f3f3d4839"
format_stamp: "Formatted at 2026-08-12 11:14:01 on dist-test-slave-drl0"
server_key: "02a0a917ec2024b1ef17dd4a76a564da"
server_key_iv: "f5d5a427fa4430f2edf106a1ea8f7246"
server_key_version: "encryptionkey@0"
I20260812 11:14:01.142401 16779 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260812 11:14:01.145840 16799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:14:01.147567 16779 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:14:01.147722 16779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "41abea7da6354051a18b687f3f3d4839"
format_stamp: "Formatted at 2026-08-12 11:14:01 on dist-test-slave-drl0"
server_key: "02a0a917ec2024b1ef17dd4a76a564da"
server_key_iv: "f5d5a427fa4430f2edf106a1ea8f7246"
server_key_version: "encryptionkey@0"
I20260812 11:14:01.147859 16779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:14:01.177438 16779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:14:01.181748 16779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:14:01.181993 16779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:14:01.190222 16779 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37201
I20260812 11:14:01.190268 16885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37201 every 8 connection(s)
I20260812 11:14:01.191332 16779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:14:01.194456 16886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:14:01.197888 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 16779
I20260812 11:14:01.198062 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:14:01.198377 17587 external_mini_cluster.cc:1428] Setting key 288a833dc60a0e9bc53df7605c8f4ef0
I20260812 11:14:01.200835 16886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41abea7da6354051a18b687f3f3d4839: Bootstrap starting.
I20260812 11:14:01.203217 16886 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41abea7da6354051a18b687f3f3d4839: Neither blocks nor log segments found. Creating new log.
I20260812 11:14:01.203999 16886 log.cc:824] T 00000000000000000000000000000000 P 41abea7da6354051a18b687f3f3d4839: Log is configured to *not* fsync() on all Append() calls
I20260812 11:14:01.206022 16886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41abea7da6354051a18b687f3f3d4839: No bootstrap required, opened a new log
Aug 12 11:14:01 dist-test-slave-drl0 krb5kdc[16761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 11:14:01.210736 16886 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41abea7da6354051a18b687f3f3d4839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41abea7da6354051a18b687f3f3d4839" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37201 } }
I20260812 11:14:01.211061 16886 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41abea7da6354051a18b687f3f3d4839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:14:01.211129 16886 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41abea7da6354051a18b687f3f3d4839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41abea7da6354051a18b687f3f3d4839, State: Initialized, Role: FOLLOWER
I20260812 11:14:01.211659 16886 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41abea7da6354051a18b687f3f3d4839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41abea7da6354051a18b687f3f3d4839" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37201 } }
I20260812 11:14:01.211791 16886 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41abea7da6354051a18b687f3f3d4839 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:14:01.211882 16886 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41abea7da6354051a18b687f3f3d4839 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:14:01.212004 16886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41abea7da6354051a18b687f3f3d4839 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:14:01.213073 16886 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41abea7da6354051a18b687f3f3d4839 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41abea7da6354051a18b687f3f3d4839" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37201 } }
I20260812 11:14:01.213517 16886 leader_election.cc:302] T 00000000000000000000000000000000 P 41abea7da6354051a18b687f3f3d4839 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41abea7da6354051a18b687f3f3d4839; no voters:
I20260812 11:14:01.213904 16886 leader_election.cc:288] T 00000000000000000000000000000000 P 41abea7da6354051a18b687f3f3d4839 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:14:01.213979 16892 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41abea7da6354051a18b687f3f3d4839 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:14:01.214226 16892 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41abea7da6354051a18b687f3f3d4839 [term 1 LEADER]: Becoming Leader. State: Replica: 41abea7da6354051a18b687f3f3d4839, State: Running, Role: LEADER
I20260812 11:14:01.214546 16892 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41abea7da6354051a18b687f3f3d4839 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41abea7da6354051a18b687f3f3d4839" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37201 } }
I20260812 11:14:01.215382 16886 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41abea7da6354051a18b687f3f3d4839 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:14:01.215880 16893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41abea7da6354051a18b687f3f3d4839 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41abea7da6354051a18b687f3f3d4839" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41abea7da6354051a18b687f3f3d4839" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37201 } } }
I20260812 11:14:01.216075 16893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41abea7da6354051a18b687f3f3d4839 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:01.216603 16894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41abea7da6354051a18b687f3f3d4839 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41abea7da6354051a18b687f3f3d4839. Latest consensus state: current_term: 1 leader_uuid: "41abea7da6354051a18b687f3f3d4839" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41abea7da6354051a18b687f3f3d4839" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37201 } } }
I20260812 11:14:01.216733 16894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41abea7da6354051a18b687f3f3d4839 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:01.216977 16896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:14:01.217365 16889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:01.200151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47555 (local address 127.0.0.1:37201)
0812 11:14:01.200794 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:01.200818 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:01.200880 (+ 62us) server_negotiation.cc:407] Connection header received
0812 11:14:01.201892 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:01.201915 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:01.202395 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:01.202855 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:01.203971 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:01.205042 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:01.205914 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:01.206444 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:01.206619 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:01.209943 (+ 3324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:01.209949 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:01.209963 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:01.209993 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:01.212026 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:01.213005 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:01.213015 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:01.213025 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:01.213265 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:01.213714 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:01.213719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:01.213721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:01.214465 (+ 744us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:01.215515 (+ 1050us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:01.215764 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 11:14:01.216081 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":134,"threads_started":1}
I20260812 11:14:01.220672 16896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:14:01.221223 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:14:01.227839 16896 catalog_manager.cc:1347] Generated new cluster ID: d439e91b49c7498381481ab6fca39bd6
I20260812 11:14:01.227916 16896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:14:01.233670 16889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:01.223485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59781 (local address 127.0.0.1:37201)
0812 11:14:01.223593 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:01.223598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:01.224124 (+ 526us) server_negotiation.cc:407] Connection header received
0812 11:14:01.224250 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:01.224254 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:01.224304 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:01.224382 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:01.225253 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:01.225937 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:01.226765 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:01.227426 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:01.227525 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:01.229156 (+ 1631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:01.229159 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:01.229162 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:01.229186 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:01.230851 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:01.231428 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:01.231432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:01.231434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:01.231514 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:01.231860 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:01.231863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:01.231865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:01.232559 (+ 694us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:01.232709 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:01.233401 (+ 692us) server_negotiation.cc:299] Negotiation successful
0812 11:14:01.233499 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 11:14:01.234370 16896 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:14:01.235425 16896 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:14:01.248786 16896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41abea7da6354051a18b687f3f3d4839: Generated new TSK 0
I20260812 11:14:01.249653 16896 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:14:01.260356 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 16779
W20260812 11:14:01.274446 16757 connection.cc:535] client connection to 127.0.0.1:37201 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37201): Connection reset by peer (error 104)
2026-08-12T11:14:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1759 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:14:01 dist-test-slave-drl0 krb5kdc[16930](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:14:01 dist-test-slave-drl0 krb5kdc[16930](info): set up 2 sockets
Aug 12 11:14:01 dist-test-slave-drl0 krb5kdc[16930](info): commencing operation
krb5kdc: starting...
W20260812 11:14:02.555037 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:14:02 dist-test-slave-drl0 krb5kdc[16930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:14:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:14:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 11:14:02.695655 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43433
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:14:02.817978 16947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:14:02.818272 16947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:14:02.818347 16947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:14:02.822647 16947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:14:02.822754 16947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:14:02.822793 16947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:14:02.822832 16947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:14:02.822865 16947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:14:02.828122 16947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43433
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:14:02.829177 16947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:14:02.830183 16947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:14:02.839915 16963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:02.839915 16957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:02.840139 16959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:02.840205 16947 server_base.cc:1034] running on GCE node
I20260812 11:14:02.841136 16947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:14:02.842536 16947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:14:02.843763 16947 hybrid_clock.cc:648] HybridClock initialized: now 1786533242843735 us; error 41 us; skew 500 ppm
Aug 12 11:14:02 dist-test-slave-drl0 krb5kdc[16930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533242, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:14:02.846921 16947 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 11:14:02.847925 16947 webserver.cc:460] Webserver started at http://127.0.0.1:41991/ using document root <none> and password file <none>
I20260812 11:14:02.848451 16947 fs_manager.cc:359] Metadata directory not provided
I20260812 11:14:02.848515 16947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:14:02.848686 16947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:14:02.850682 16947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "b5796d2bb6e94b2c878f9770585dab9b"
format_stamp: "Formatted at 2026-08-12 11:14:02 on dist-test-slave-drl0"
server_key: "4eaf14ab1a54733419d68eee803baaa5"
server_key_iv: "f910224e3dd06a6e3eb4ffa9afa99adb"
server_key_version: "encryptionkey@0"
I20260812 11:14:02.851416 16947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "b5796d2bb6e94b2c878f9770585dab9b"
format_stamp: "Formatted at 2026-08-12 11:14:02 on dist-test-slave-drl0"
server_key: "4eaf14ab1a54733419d68eee803baaa5"
server_key_iv: "f910224e3dd06a6e3eb4ffa9afa99adb"
server_key_version: "encryptionkey@0"
I20260812 11:14:02.855841 16947 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 11:14:02.859442 16973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:14:02.860978 16947 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260812 11:14:02.861133 16947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "b5796d2bb6e94b2c878f9770585dab9b"
format_stamp: "Formatted at 2026-08-12 11:14:02 on dist-test-slave-drl0"
server_key: "4eaf14ab1a54733419d68eee803baaa5"
server_key_iv: "f910224e3dd06a6e3eb4ffa9afa99adb"
server_key_version: "encryptionkey@0"
I20260812 11:14:02.861248 16947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:14:02.889576 16947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:14:02.893042 16947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:14:02.893272 16947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:14:02.902840 16947 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43433
I20260812 11:14:02.902900 17062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43433 every 8 connection(s)
I20260812 11:14:02.903991 16947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:14:02.906875 17065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:14:02.912631 17065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5796d2bb6e94b2c878f9770585dab9b: Bootstrap starting.
I20260812 11:14:02.913784 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 16947
I20260812 11:14:02.913962 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:14:02.914429 17587 external_mini_cluster.cc:1428] Setting key 64853e81307e591e33fca4c4aa11808f
I20260812 11:14:02.915387 17065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5796d2bb6e94b2c878f9770585dab9b: Neither blocks nor log segments found. Creating new log.
I20260812 11:14:02.916138 17065 log.cc:824] T 00000000000000000000000000000000 P b5796d2bb6e94b2c878f9770585dab9b: Log is configured to *not* fsync() on all Append() calls
I20260812 11:14:02.918385 17065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5796d2bb6e94b2c878f9770585dab9b: No bootstrap required, opened a new log
I20260812 11:14:02.922870 17065 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5796d2bb6e94b2c878f9770585dab9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5796d2bb6e94b2c878f9770585dab9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43433 } }
I20260812 11:14:02.923180 17065 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5796d2bb6e94b2c878f9770585dab9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:14:02.923244 17065 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5796d2bb6e94b2c878f9770585dab9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5796d2bb6e94b2c878f9770585dab9b, State: Initialized, Role: FOLLOWER
I20260812 11:14:02.923651 17065 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5796d2bb6e94b2c878f9770585dab9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5796d2bb6e94b2c878f9770585dab9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43433 } }
I20260812 11:14:02.923812 17065 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5796d2bb6e94b2c878f9770585dab9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:14:02.923892 17065 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5796d2bb6e94b2c878f9770585dab9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:14:02.924009 17065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5796d2bb6e94b2c878f9770585dab9b [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:14:02.924957 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:02.915650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39615 (local address 127.0.0.1:43433)
0812 11:14:02.916230 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:02.916247 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:02.916307 (+ 60us) server_negotiation.cc:407] Connection header received
0812 11:14:02.917157 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:02.917176 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:02.917494 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:02.917759 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:02.918220 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:02.919532 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:02.920895 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:02.921674 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:02.921873 (+ 199us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:02.922855 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:02.922860 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:02.922873 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:14:02.923580 (+ 707us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:02.923637 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:14:02.923662 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:02.923803 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:02.924055 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 11:14:02.924208 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":178,"threads_started":1}
I20260812 11:14:02.924943 17065 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5796d2bb6e94b2c878f9770585dab9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5796d2bb6e94b2c878f9770585dab9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43433 } }
I20260812 11:14:02.925392 17065 leader_election.cc:302] T 00000000000000000000000000000000 P b5796d2bb6e94b2c878f9770585dab9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5796d2bb6e94b2c878f9770585dab9b; no voters:
I20260812 11:14:02.925745 17065 leader_election.cc:288] T 00000000000000000000000000000000 P b5796d2bb6e94b2c878f9770585dab9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:14:02.925822 17074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5796d2bb6e94b2c878f9770585dab9b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:14:02.926033 17074 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5796d2bb6e94b2c878f9770585dab9b [term 1 LEADER]: Becoming Leader. State: Replica: b5796d2bb6e94b2c878f9770585dab9b, State: Running, Role: LEADER
I20260812 11:14:02.926332 17074 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5796d2bb6e94b2c878f9770585dab9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5796d2bb6e94b2c878f9770585dab9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43433 } }
I20260812 11:14:02.927071 17065 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5796d2bb6e94b2c878f9770585dab9b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:14:02.927522 17074 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5796d2bb6e94b2c878f9770585dab9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5796d2bb6e94b2c878f9770585dab9b. Latest consensus state: current_term: 1 leader_uuid: "b5796d2bb6e94b2c878f9770585dab9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5796d2bb6e94b2c878f9770585dab9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43433 } } }
I20260812 11:14:02.927636 17074 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5796d2bb6e94b2c878f9770585dab9b [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:02.927845 17074 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5796d2bb6e94b2c878f9770585dab9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5796d2bb6e94b2c878f9770585dab9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5796d2bb6e94b2c878f9770585dab9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43433 } } }
I20260812 11:14:02.927950 17074 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5796d2bb6e94b2c878f9770585dab9b [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:02.928058 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:14:02.929726 17081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:14:02.933604 17081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:14:02.938637 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:02.933305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52901 (local address 127.0.0.1:43433)
0812 11:14:02.933444 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:02.933449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:02.933735 (+ 286us) server_negotiation.cc:407] Connection header received
0812 11:14:02.933941 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:02.933948 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:02.934006 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:02.934080 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:02.934526 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:02.935323 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:02.936547 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:02.936952 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:02.937017 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:02.938063 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:02.938066 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:02.938069 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:14:02.938081 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:02.938110 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:14:02.938123 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:02.938189 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:02.938452 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 11:14:02.938486 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 11:14:02.940848 17081 catalog_manager.cc:1347] Generated new cluster ID: 2ba71a26882a46629d873d92401db2eb
I20260812 11:14:02.940920 17081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:14:02.953749 17081 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:14:02.955192 17081 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:14:02.963704 17081 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5796d2bb6e94b2c878f9770585dab9b: Generated new TSK 0
I20260812 11:14:02.964509 17081 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:14:02.973454 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 16947
W20260812 11:14:02.994232 16926 connection.cc:535] client connection to 127.0.0.1:43433 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43433): Connection reset by peer (error 104)
2026-08-12T11:14:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1719 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:14:03 dist-test-slave-drl0 krb5kdc[17117](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:14:03 dist-test-slave-drl0 krb5kdc[17117](info): set up 2 sockets
Aug 12 11:14:03 dist-test-slave-drl0 krb5kdc[17117](info): commencing operation
krb5kdc: starting...
W20260812 11:14:04.292227 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.001s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:14:04 dist-test-slave-drl0 krb5kdc[17117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:14:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:14:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 11:14:04.434382 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40379
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:14:04.563424 17133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:14:04.563716 17133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:14:04.563766 17133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:14:04.568616 17133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:14:04.568684 17133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:14:04.568704 17133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:14:04.568722 17133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:14:04.568737 17133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:14:04.574385 17133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40379
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:14:04.575484 17133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:14:04.577461 17133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:14:04.588042 17144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:04.588109 17133 server_base.cc:1034] running on GCE node
W20260812 11:14:04.588049 17145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:04.588063 17147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:04.588897 17133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:14:04.590059 17133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:14:04.591271 17133 hybrid_clock.cc:648] HybridClock initialized: now 1786533244591242 us; error 63 us; skew 500 ppm
Aug 12 11:14:04 dist-test-slave-drl0 krb5kdc[17117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533244, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:14:04.594439 17133 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 11:14:04.595599 17133 webserver.cc:460] Webserver started at http://127.0.0.1:41201/ using document root <none> and password file <none>
I20260812 11:14:04.596469 17133 fs_manager.cc:359] Metadata directory not provided
I20260812 11:14:04.596546 17133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:14:04.596788 17133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:14:04.599244 17133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "86e48a4acd694456b73109388461e234"
format_stamp: "Formatted at 2026-08-12 11:14:04 on dist-test-slave-drl0"
server_key: "08f8d1dd710b4242d1e49930fde8fa46"
server_key_iv: "afd47a7e5c67e5c125d4f970562dd2f6"
server_key_version: "encryptionkey@0"
I20260812 11:14:04.599830 17133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "86e48a4acd694456b73109388461e234"
format_stamp: "Formatted at 2026-08-12 11:14:04 on dist-test-slave-drl0"
server_key: "08f8d1dd710b4242d1e49930fde8fa46"
server_key_iv: "afd47a7e5c67e5c125d4f970562dd2f6"
server_key_version: "encryptionkey@0"
I20260812 11:14:04.605436 17133 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260812 11:14:04.608544 17156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:14:04.610317 17133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 11:14:04.610641 17133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "86e48a4acd694456b73109388461e234"
format_stamp: "Formatted at 2026-08-12 11:14:04 on dist-test-slave-drl0"
server_key: "08f8d1dd710b4242d1e49930fde8fa46"
server_key_iv: "afd47a7e5c67e5c125d4f970562dd2f6"
server_key_version: "encryptionkey@0"
I20260812 11:14:04.610786 17133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:14:04.616914 17133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:14:04.620786 17133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:14:04.621033 17133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:14:04.629120 17133 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40379
I20260812 11:14:04.629145 17238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40379 every 8 connection(s)
I20260812 11:14:04.630059 17133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:14:04.632113 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 17133
I20260812 11:14:04.632292 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:14:04.632606 17587 external_mini_cluster.cc:1428] Setting key 22d2fbf75b216868fbceb31ad7c2d06c
I20260812 11:14:04.633343 17239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:14:04.638289 17246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:04.634507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48581 (local address 127.0.0.1:40379)
0812 11:14:04.634961 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:04.634993 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:04.635031 (+ 38us) server_negotiation.cc:407] Connection header received
0812 11:14:04.635678 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:04.635694 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:04.635961 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:04.636230 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:04.637080 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:04.637085 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:04.637097 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:14:04.637109 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:04.637170 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:14:04.637191 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:04.637305 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:04.637436 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 11:14:04.637607 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":159,"threads_started":1}
I20260812 11:14:04.640765 17239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86e48a4acd694456b73109388461e234: Bootstrap starting.
I20260812 11:14:04.642051 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:14:04.643319 17239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86e48a4acd694456b73109388461e234: Neither blocks nor log segments found. Creating new log.
I20260812 11:14:04.644044 17239 log.cc:824] T 00000000000000000000000000000000 P 86e48a4acd694456b73109388461e234: Log is configured to *not* fsync() on all Append() calls
I20260812 11:14:04.646379 17246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:04.644263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57857 (local address 127.0.0.1:40379)
0812 11:14:04.644403 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:04.644407 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:04.644629 (+ 222us) server_negotiation.cc:407] Connection header received
0812 11:14:04.644707 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:04.644710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:04.644743 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:04.644858 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:04.645753 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:04.645756 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:04.645758 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:14:04.645769 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:04.645800 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:14:04.645811 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:04.645871 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:04.646067 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 11:14:04.646223 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 11:14:04.646605 17239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86e48a4acd694456b73109388461e234: No bootstrap required, opened a new log
I20260812 11:14:04.650108 17239 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86e48a4acd694456b73109388461e234 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e48a4acd694456b73109388461e234" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40379 } }
I20260812 11:14:04.650355 17239 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86e48a4acd694456b73109388461e234 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:14:04.650446 17239 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86e48a4acd694456b73109388461e234 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86e48a4acd694456b73109388461e234, State: Initialized, Role: FOLLOWER
I20260812 11:14:04.650867 17239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86e48a4acd694456b73109388461e234 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e48a4acd694456b73109388461e234" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40379 } }
I20260812 11:14:04.651077 17239 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86e48a4acd694456b73109388461e234 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:14:04.651166 17239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86e48a4acd694456b73109388461e234 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:14:04.651264 17239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86e48a4acd694456b73109388461e234 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:14:04.652120 17239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86e48a4acd694456b73109388461e234 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e48a4acd694456b73109388461e234" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40379 } }
I20260812 11:14:04.652469 17239 leader_election.cc:302] T 00000000000000000000000000000000 P 86e48a4acd694456b73109388461e234 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86e48a4acd694456b73109388461e234; no voters:
I20260812 11:14:04.652829 17239 leader_election.cc:288] T 00000000000000000000000000000000 P 86e48a4acd694456b73109388461e234 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:14:04.652907 17255 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86e48a4acd694456b73109388461e234 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:14:04.653570 17255 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86e48a4acd694456b73109388461e234 [term 1 LEADER]: Becoming Leader. State: Replica: 86e48a4acd694456b73109388461e234, State: Running, Role: LEADER
I20260812 11:14:04.653860 17255 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86e48a4acd694456b73109388461e234 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e48a4acd694456b73109388461e234" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40379 } }
I20260812 11:14:04.654580 17239 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86e48a4acd694456b73109388461e234 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:14:04.655652 17259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86e48a4acd694456b73109388461e234 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86e48a4acd694456b73109388461e234. Latest consensus state: current_term: 1 leader_uuid: "86e48a4acd694456b73109388461e234" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e48a4acd694456b73109388461e234" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40379 } } }
I20260812 11:14:04.655701 17262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86e48a4acd694456b73109388461e234 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86e48a4acd694456b73109388461e234" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e48a4acd694456b73109388461e234" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40379 } } }
I20260812 11:14:04.655781 17259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86e48a4acd694456b73109388461e234 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:04.655798 17262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86e48a4acd694456b73109388461e234 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:04.656158 17271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:14:04.659358 17271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:14:04.666445 17271 catalog_manager.cc:1347] Generated new cluster ID: e445673b85124e8d9d673827e9f6c0fb
I20260812 11:14:04.666622 17271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:14:04.682399 17271 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:14:04.682767 17271 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:14:04.689429 17271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86e48a4acd694456b73109388461e234: Generated new TSK 0
I20260812 11:14:04.690114 17271 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:14:04.696971 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 17133
2026-08-12T11:14:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1712 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:14:04 dist-test-slave-drl0 krb5kdc[17295](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:14:04 dist-test-slave-drl0 krb5kdc[17295](info): set up 2 sockets
Aug 12 11:14:04 dist-test-slave-drl0 krb5kdc[17295](info): commencing operation
krb5kdc: starting...
W20260812 11:14:06.024551 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:14:06 dist-test-slave-drl0 krb5kdc[17295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533246, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:14:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:14:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 11:14:06.163010 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42435
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:14:06.283260 17310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:14:06.283535 17310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:14:06.283576 17310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:14:06.287518 17310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:14:06.287572 17310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:14:06.287592 17310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:14:06.287609 17310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:14:06.287624 17310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:14:06.292851 17310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:14:06.293877 17310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:14:06.294901 17310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:14:06.303731 17325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:06.303756 17323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:06.303756 17322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:06.303937 17310 server_base.cc:1034] running on GCE node
I20260812 11:14:06.304787 17310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:14:06.306012 17310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:14:06.307283 17310 hybrid_clock.cc:648] HybridClock initialized: now 1786533246307261 us; error 46 us; skew 500 ppm
Aug 12 11:14:06 dist-test-slave-drl0 krb5kdc[17295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533246, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:14:06.316463 17310 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 11:14:06.317544 17310 webserver.cc:460] Webserver started at http://127.0.0.1:37421/ using document root <none> and password file <none>
I20260812 11:14:06.318099 17310 fs_manager.cc:359] Metadata directory not provided
I20260812 11:14:06.318161 17310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:14:06.318331 17310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:14:06.320472 17310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "43cb7394059744f69edf463cba371bf2"
format_stamp: "Formatted at 2026-08-12 11:14:06 on dist-test-slave-drl0"
server_key: "24ab47c81e311e8f9d7d40f402fae43f"
server_key_iv: "f0e452fcc2ecb5bfc0f90564f1b4248a"
server_key_version: "encryptionkey@0"
I20260812 11:14:06.321014 17310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "43cb7394059744f69edf463cba371bf2"
format_stamp: "Formatted at 2026-08-12 11:14:06 on dist-test-slave-drl0"
server_key: "24ab47c81e311e8f9d7d40f402fae43f"
server_key_iv: "f0e452fcc2ecb5bfc0f90564f1b4248a"
server_key_version: "encryptionkey@0"
I20260812 11:14:06.324996 17310 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 11:14:06.327576 17336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:14:06.328879 17310 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:14:06.328995 17310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "43cb7394059744f69edf463cba371bf2"
format_stamp: "Formatted at 2026-08-12 11:14:06 on dist-test-slave-drl0"
server_key: "24ab47c81e311e8f9d7d40f402fae43f"
server_key_iv: "f0e452fcc2ecb5bfc0f90564f1b4248a"
server_key_version: "encryptionkey@0"
I20260812 11:14:06.329294 17310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:14:06.345934 17310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:14:06.349337 17310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:14:06.349553 17310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:14:06.359105 17310 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42435
I20260812 11:14:06.359143 17430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42435 every 8 connection(s)
I20260812 11:14:06.360184 17310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:14:06.363219 17434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:14:06.368649 17434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43cb7394059744f69edf463cba371bf2: Bootstrap starting.
I20260812 11:14:06.370366 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 17310
I20260812 11:14:06.370522 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:14:06.370816 17587 external_mini_cluster.cc:1428] Setting key 0e816de2341b34a5b7576ade28d0ce15
I20260812 11:14:06.371287 17434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43cb7394059744f69edf463cba371bf2: Neither blocks nor log segments found. Creating new log.
I20260812 11:14:06.372437 17434 log.cc:824] T 00000000000000000000000000000000 P 43cb7394059744f69edf463cba371bf2: Log is configured to *not* fsync() on all Append() calls
I20260812 11:14:06.374341 17434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43cb7394059744f69edf463cba371bf2: No bootstrap required, opened a new log
I20260812 11:14:06.377749 17434 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43cb7394059744f69edf463cba371bf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43cb7394059744f69edf463cba371bf2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42435 } }
I20260812 11:14:06.377997 17434 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43cb7394059744f69edf463cba371bf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:14:06.378077 17434 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43cb7394059744f69edf463cba371bf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cb7394059744f69edf463cba371bf2, State: Initialized, Role: FOLLOWER
I20260812 11:14:06.378566 17434 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43cb7394059744f69edf463cba371bf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43cb7394059744f69edf463cba371bf2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42435 } }
I20260812 11:14:06.378723 17434 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43cb7394059744f69edf463cba371bf2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:14:06.378813 17434 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43cb7394059744f69edf463cba371bf2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:14:06.378911 17434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43cb7394059744f69edf463cba371bf2 [term 0 FOLLOWER]: Advancing to term 1
Aug 12 11:14:06 dist-test-slave-drl0 krb5kdc[17295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533246, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 11:14:06.379850 17434 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43cb7394059744f69edf463cba371bf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43cb7394059744f69edf463cba371bf2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42435 } }
I20260812 11:14:06.380191 17434 leader_election.cc:302] T 00000000000000000000000000000000 P 43cb7394059744f69edf463cba371bf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43cb7394059744f69edf463cba371bf2; no voters:
I20260812 11:14:06.380506 17434 leader_election.cc:288] T 00000000000000000000000000000000 P 43cb7394059744f69edf463cba371bf2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:14:06.380686 17443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43cb7394059744f69edf463cba371bf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:14:06.381177 17443 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43cb7394059744f69edf463cba371bf2 [term 1 LEADER]: Becoming Leader. State: Replica: 43cb7394059744f69edf463cba371bf2, State: Running, Role: LEADER
I20260812 11:14:06.381515 17443 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43cb7394059744f69edf463cba371bf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43cb7394059744f69edf463cba371bf2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42435 } }
I20260812 11:14:06.382053 17434 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43cb7394059744f69edf463cba371bf2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:14:06.383229 17446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43cb7394059744f69edf463cba371bf2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43cb7394059744f69edf463cba371bf2. Latest consensus state: current_term: 1 leader_uuid: "43cb7394059744f69edf463cba371bf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43cb7394059744f69edf463cba371bf2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42435 } } }
I20260812 11:14:06.383368 17446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43cb7394059744f69edf463cba371bf2 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:06.383677 17444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43cb7394059744f69edf463cba371bf2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43cb7394059744f69edf463cba371bf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43cb7394059744f69edf463cba371bf2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42435 } } }
I20260812 11:14:06.383807 17444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43cb7394059744f69edf463cba371bf2 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:06.386265 17438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:06.371988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53561 (local address 127.0.0.1:42435)
0812 11:14:06.372552 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:06.372572 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:06.372646 (+ 74us) server_negotiation.cc:407] Connection header received
0812 11:14:06.373311 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:06.373328 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:06.373644 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:06.373983 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:06.374819 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:06.375822 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:06.376679 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:06.377146 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:06.377320 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:06.379966 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:06.379971 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:06.380008 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:06.380041 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:06.382321 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:06.382855 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:06.382861 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:06.382868 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:06.383032 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:06.383491 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:06.383494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:06.383496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:06.384084 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:06.384994 (+ 910us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:06.385149 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 11:14:06.385479 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":160,"threads_started":1}
W20260812 11:14:06.387392 17474 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 43cb7394059744f69edf463cba371bf2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:14:06.387468 17474 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:14:06.387609 17458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:14:06.388554 17458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:14:06.389458 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:14:06.394215 17458 catalog_manager.cc:1347] Generated new cluster ID: bc0ea4f364be4ed1ace665ba080c24d8
I20260812 11:14:06.394287 17458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:14:06.400293 17438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:06.391787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60445 (local address 127.0.0.1:42435)
0812 11:14:06.391891 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:06.391896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:06.392132 (+ 236us) server_negotiation.cc:407] Connection header received
0812 11:14:06.392342 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:06.392345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:06.392392 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:06.392482 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:06.393668 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:06.394402 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:06.395309 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:06.395570 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:06.395687 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:06.397020 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:06.397024 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:06.397026 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:06.397057 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:06.398604 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:06.399065 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:06.399069 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:06.399070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:06.399159 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:06.399445 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:06.399448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:06.399450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:06.399624 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:06.399764 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:06.399993 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 11:14:06.400107 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":42}
I20260812 11:14:06.409922 17458 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:14:06.411131 17458 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:14:06.416754 17458 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43cb7394059744f69edf463cba371bf2: Generated new TSK 0
I20260812 11:14:06.417402 17458 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:14:06.425531 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 17310
2026-08-12T11:14:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1730 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:14:06 dist-test-slave-drl0 krb5kdc[17491](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:14:06 dist-test-slave-drl0 krb5kdc[17491](info): set up 2 sockets
Aug 12 11:14:06 dist-test-slave-drl0 krb5kdc[17491](info): commencing operation
krb5kdc: starting...
W20260812 11:14:07.729540 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.277s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:14:07 dist-test-slave-drl0 krb5kdc[17491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:14:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:14:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 11:14:07.867067 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40967
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:14:07.988022 17513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:14:07.988394 17513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:14:07.988482 17513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:14:07.994297 17513 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:14:07.994509 17513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:14:07.994580 17513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:14:07.994617 17513 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:14:07.994745 17513 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:14:08.001765 17513 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:14:08.002885 17513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:14:08.004132 17513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:14:08.013118 17521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:08.013106 17522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:08.013166 17526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:08.013552 17513 server_base.cc:1034] running on GCE node
I20260812 11:14:08.014143 17513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:14:08.015595 17513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:14:08.016963 17513 hybrid_clock.cc:648] HybridClock initialized: now 1786533248016906 us; error 74 us; skew 500 ppm
Aug 12 11:14:08 dist-test-slave-drl0 krb5kdc[17491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533248, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:14:08.020457 17513 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 11:14:08.021338 17513 webserver.cc:460] Webserver started at http://127.0.0.1:43809/ using document root <none> and password file <none>
I20260812 11:14:08.021869 17513 fs_manager.cc:359] Metadata directory not provided
I20260812 11:14:08.021938 17513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:14:08.022104 17513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:14:08.024183 17513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "f9cfa5ab7bf4473fb5c65fa83bb0a8ce"
format_stamp: "Formatted at 2026-08-12 11:14:08 on dist-test-slave-drl0"
server_key: "1eb8421c7ba2eed4e4b44c443e01f3ac"
server_key_iv: "d482f4d7460f0376b957613734ddcc2c"
server_key_version: "encryptionkey@0"
I20260812 11:14:08.024698 17513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "f9cfa5ab7bf4473fb5c65fa83bb0a8ce"
format_stamp: "Formatted at 2026-08-12 11:14:08 on dist-test-slave-drl0"
server_key: "1eb8421c7ba2eed4e4b44c443e01f3ac"
server_key_iv: "d482f4d7460f0376b957613734ddcc2c"
server_key_version: "encryptionkey@0"
I20260812 11:14:08.028364 17513 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:14:08.031672 17539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:14:08.033247 17513 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:14:08.033380 17513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "f9cfa5ab7bf4473fb5c65fa83bb0a8ce"
format_stamp: "Formatted at 2026-08-12 11:14:08 on dist-test-slave-drl0"
server_key: "1eb8421c7ba2eed4e4b44c443e01f3ac"
server_key_iv: "d482f4d7460f0376b957613734ddcc2c"
server_key_version: "encryptionkey@0"
I20260812 11:14:08.033468 17513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:14:08.069949 17513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:14:08.074107 17513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:14:08.074296 17513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:14:08.082069 17513 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40967
I20260812 11:14:08.082368 17639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40967 every 8 connection(s)
I20260812 11:14:08.083130 17513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:14:08.084878 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 17513
I20260812 11:14:08.085049 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:14:08.085389 17587 external_mini_cluster.cc:1428] Setting key 349268365188c4fece9e666e142bd986
I20260812 11:14:08.086639 17640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:14:08.093538 17640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9cfa5ab7bf4473fb5c65fa83bb0a8ce: Bootstrap starting.
I20260812 11:14:08.095799 17645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:08.087169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52021 (local address 127.0.0.1:40967)
0812 11:14:08.087779 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:08.087805 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:08.087857 (+ 52us) server_negotiation.cc:407] Connection header received
0812 11:14:08.088688 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:08.088708 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:08.089050 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:08.089345 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:08.089741 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:08.090763 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:08.092086 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:08.092811 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:08.092954 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:08.093873 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:08.093878 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:08.093891 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:14:08.094313 (+ 422us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:08.094370 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:14:08.094394 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:08.094492 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:08.094728 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 11:14:08.095033 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":222,"threads_started":1}
I20260812 11:14:08.096069 17640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9cfa5ab7bf4473fb5c65fa83bb0a8ce: Neither blocks nor log segments found. Creating new log.
I20260812 11:14:08.096809 17640 log.cc:824] T 00000000000000000000000000000000 P f9cfa5ab7bf4473fb5c65fa83bb0a8ce: Log is configured to *not* fsync() on all Append() calls
I20260812 11:14:08.098786 17640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9cfa5ab7bf4473fb5c65fa83bb0a8ce: No bootstrap required, opened a new log
I20260812 11:14:08.098798 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:14:08.103777 17640 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9cfa5ab7bf4473fb5c65fa83bb0a8ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9cfa5ab7bf4473fb5c65fa83bb0a8ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40967 } }
I20260812 11:14:08.104378 17640 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9cfa5ab7bf4473fb5c65fa83bb0a8ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:14:08.104461 17640 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9cfa5ab7bf4473fb5c65fa83bb0a8ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9cfa5ab7bf4473fb5c65fa83bb0a8ce, State: Initialized, Role: FOLLOWER
I20260812 11:14:08.104930 17640 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9cfa5ab7bf4473fb5c65fa83bb0a8ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9cfa5ab7bf4473fb5c65fa83bb0a8ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40967 } }
I20260812 11:14:08.105139 17640 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9cfa5ab7bf4473fb5c65fa83bb0a8ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:14:08.105219 17640 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9cfa5ab7bf4473fb5c65fa83bb0a8ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:14:08.105341 17640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9cfa5ab7bf4473fb5c65fa83bb0a8ce [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:14:08.105999 17645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:08.101234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44943 (local address 127.0.0.1:40967)
0812 11:14:08.101432 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:08.101437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:08.101458 (+ 21us) server_negotiation.cc:407] Connection header received
0812 11:14:08.101555 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:08.101560 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:08.101609 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:08.101698 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:08.101988 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:08.102564 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:08.103603 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:08.103856 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:08.104462 (+ 606us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:08.105364 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:08.105373 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:08.105376 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:14:08.105387 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:08.105411 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:14:08.105422 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:08.105493 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:08.105632 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 11:14:08.105678 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 11:14:08.106467 17640 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9cfa5ab7bf4473fb5c65fa83bb0a8ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9cfa5ab7bf4473fb5c65fa83bb0a8ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40967 } }
I20260812 11:14:08.106806 17640 leader_election.cc:302] T 00000000000000000000000000000000 P f9cfa5ab7bf4473fb5c65fa83bb0a8ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9cfa5ab7bf4473fb5c65fa83bb0a8ce; no voters:
I20260812 11:14:08.107133 17640 leader_election.cc:288] T 00000000000000000000000000000000 P f9cfa5ab7bf4473fb5c65fa83bb0a8ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:14:08.107486 17660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9cfa5ab7bf4473fb5c65fa83bb0a8ce [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:14:08.107971 17660 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9cfa5ab7bf4473fb5c65fa83bb0a8ce [term 1 LEADER]: Becoming Leader. State: Replica: f9cfa5ab7bf4473fb5c65fa83bb0a8ce, State: Running, Role: LEADER
I20260812 11:14:08.108261 17660 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9cfa5ab7bf4473fb5c65fa83bb0a8ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9cfa5ab7bf4473fb5c65fa83bb0a8ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40967 } }
I20260812 11:14:08.108436 17640 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9cfa5ab7bf4473fb5c65fa83bb0a8ce [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:14:08.109752 17661 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9cfa5ab7bf4473fb5c65fa83bb0a8ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9cfa5ab7bf4473fb5c65fa83bb0a8ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9cfa5ab7bf4473fb5c65fa83bb0a8ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40967 } } }
I20260812 11:14:08.109910 17661 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9cfa5ab7bf4473fb5c65fa83bb0a8ce [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:08.110343 17662 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9cfa5ab7bf4473fb5c65fa83bb0a8ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9cfa5ab7bf4473fb5c65fa83bb0a8ce. Latest consensus state: current_term: 1 leader_uuid: "f9cfa5ab7bf4473fb5c65fa83bb0a8ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9cfa5ab7bf4473fb5c65fa83bb0a8ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40967 } } }
I20260812 11:14:08.110440 17685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:14:08.110440 17662 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9cfa5ab7bf4473fb5c65fa83bb0a8ce [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:08.113061 17685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:14:08.118824 17685 catalog_manager.cc:1347] Generated new cluster ID: c0ad62682e1541d1a003cdce90b09d28
I20260812 11:14:08.118916 17685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:14:08.138406 17685 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:14:08.139819 17685 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:14:08.147290 17685 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9cfa5ab7bf4473fb5c65fa83bb0a8ce: Generated new TSK 0
I20260812 11:14:08.147967 17685 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:14:08.157131 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 17513
2026-08-12T11:14:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1744 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:14:08 dist-test-slave-drl0 krb5kdc[17703](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:14:08 dist-test-slave-drl0 krb5kdc[17703](info): set up 2 sockets
Aug 12 11:14:08 dist-test-slave-drl0 krb5kdc[17703](info): commencing operation
krb5kdc: starting...
W20260812 11:14:09.480363 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:14:09 dist-test-slave-drl0 krb5kdc[17703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:14:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:14:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 11:14:09.622551 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44135
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:14:09.750171 17723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:14:09.750444 17723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:14:09.750491 17723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:14:09.754607 17723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:14:09.754900 17723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:14:09.755051 17723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:14:09.755076 17723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:14:09.755095 17723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:14:09.762523 17723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:14:09.764230 17723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:14:09.765866 17723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:14:09.777258 17729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:09.777256 17731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:09.777330 17735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:09.777742 17723 server_base.cc:1034] running on GCE node
I20260812 11:14:09.778383 17723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:14:09.779665 17723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:14:09.780937 17723 hybrid_clock.cc:648] HybridClock initialized: now 1786533249780887 us; error 42 us; skew 500 ppm
Aug 12 11:14:09 dist-test-slave-drl0 krb5kdc[17703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533249, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:14:09.784766 17723 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 11:14:09.785882 17723 webserver.cc:460] Webserver started at http://127.0.0.1:45819/ using document root <none> and password file <none>
I20260812 11:14:09.786522 17723 fs_manager.cc:359] Metadata directory not provided
I20260812 11:14:09.786620 17723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:14:09.786870 17723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:14:09.789042 17723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "627c3675178848a3a895c3b6797de531"
format_stamp: "Formatted at 2026-08-12 11:14:09 on dist-test-slave-drl0"
server_key: "eceeabd04fc9491bcabf782b5bbe6aca"
server_key_iv: "1281af5bac57e44410d72917568a66d2"
server_key_version: "encryptionkey@0"
I20260812 11:14:09.789629 17723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "627c3675178848a3a895c3b6797de531"
format_stamp: "Formatted at 2026-08-12 11:14:09 on dist-test-slave-drl0"
server_key: "eceeabd04fc9491bcabf782b5bbe6aca"
server_key_iv: "1281af5bac57e44410d72917568a66d2"
server_key_version: "encryptionkey@0"
I20260812 11:14:09.794253 17723 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 11:14:09.797183 17746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:14:09.799043 17723 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:14:09.799194 17723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "627c3675178848a3a895c3b6797de531"
format_stamp: "Formatted at 2026-08-12 11:14:09 on dist-test-slave-drl0"
server_key: "eceeabd04fc9491bcabf782b5bbe6aca"
server_key_iv: "1281af5bac57e44410d72917568a66d2"
server_key_version: "encryptionkey@0"
I20260812 11:14:09.799319 17723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:14:09.804283 17723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:14:09.807595 17723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:14:09.807785 17723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:14:09.817500 17723 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44135
I20260812 11:14:09.817508 17829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44135 every 8 connection(s)
I20260812 11:14:09.818486 17723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:14:09.820402 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 17723
I20260812 11:14:09.820559 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:14:09.820864 17587 external_mini_cluster.cc:1428] Setting key c6c481fa65e36331e0955201719440e0
I20260812 11:14:09.821877 17830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:14:09.827569 17834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:09.823049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37051 (local address 127.0.0.1:44135)
0812 11:14:09.823487 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:09.823505 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:09.823542 (+ 37us) server_negotiation.cc:407] Connection header received
0812 11:14:09.824505 (+ 963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:09.824525 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:09.824813 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:09.825104 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:09.826239 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:09.826244 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:09.826261 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:14:09.826273 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:09.826331 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:14:09.826350 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:09.826468 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:09.826601 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 11:14:09.826786 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":148,"threads_started":1}
I20260812 11:14:09.829967 17830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 627c3675178848a3a895c3b6797de531: Bootstrap starting.
I20260812 11:14:09.830328 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:14:09.832787 17830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 627c3675178848a3a895c3b6797de531: Neither blocks nor log segments found. Creating new log.
I20260812 11:14:09.833842 17830 log.cc:824] T 00000000000000000000000000000000 P 627c3675178848a3a895c3b6797de531: Log is configured to *not* fsync() on all Append() calls
I20260812 11:14:09.835623 17834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:09.832703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34541 (local address 127.0.0.1:44135)
0812 11:14:09.832842 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:09.832846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:09.833059 (+ 213us) server_negotiation.cc:407] Connection header received
0812 11:14:09.833247 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:09.833254 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:09.833297 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:09.833388 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:09.834586 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:09.834590 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:09.834592 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:14:09.834604 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:09.834635 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:14:09.834645 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:09.835216 (+ 571us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:09.835351 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 11:14:09.835417 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 11:14:09.835851 17830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 627c3675178848a3a895c3b6797de531: No bootstrap required, opened a new log
I20260812 11:14:09.839445 17830 raft_consensus.cc:348] T 00000000000000000000000000000000 P 627c3675178848a3a895c3b6797de531 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "627c3675178848a3a895c3b6797de531" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44135 } }
I20260812 11:14:09.839692 17830 raft_consensus.cc:374] T 00000000000000000000000000000000 P 627c3675178848a3a895c3b6797de531 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:14:09.839771 17830 raft_consensus.cc:729] T 00000000000000000000000000000000 P 627c3675178848a3a895c3b6797de531 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 627c3675178848a3a895c3b6797de531, State: Initialized, Role: FOLLOWER
I20260812 11:14:09.840202 17830 consensus_queue.cc:260] T 00000000000000000000000000000000 P 627c3675178848a3a895c3b6797de531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "627c3675178848a3a895c3b6797de531" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44135 } }
I20260812 11:14:09.840363 17830 raft_consensus.cc:388] T 00000000000000000000000000000000 P 627c3675178848a3a895c3b6797de531 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:14:09.840453 17830 raft_consensus.cc:482] T 00000000000000000000000000000000 P 627c3675178848a3a895c3b6797de531 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:14:09.840566 17830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 627c3675178848a3a895c3b6797de531 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:14:09.841485 17830 raft_consensus.cc:504] T 00000000000000000000000000000000 P 627c3675178848a3a895c3b6797de531 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "627c3675178848a3a895c3b6797de531" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44135 } }
I20260812 11:14:09.841823 17830 leader_election.cc:302] T 00000000000000000000000000000000 P 627c3675178848a3a895c3b6797de531 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 627c3675178848a3a895c3b6797de531; no voters:
I20260812 11:14:09.842193 17830 leader_election.cc:288] T 00000000000000000000000000000000 P 627c3675178848a3a895c3b6797de531 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:14:09.842391 17844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 627c3675178848a3a895c3b6797de531 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:14:09.842741 17844 raft_consensus.cc:686] T 00000000000000000000000000000000 P 627c3675178848a3a895c3b6797de531 [term 1 LEADER]: Becoming Leader. State: Replica: 627c3675178848a3a895c3b6797de531, State: Running, Role: LEADER
I20260812 11:14:09.843178 17844 consensus_queue.cc:237] T 00000000000000000000000000000000 P 627c3675178848a3a895c3b6797de531 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "627c3675178848a3a895c3b6797de531" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44135 } }
I20260812 11:14:09.843660 17830 sys_catalog.cc:564] T 00000000000000000000000000000000 P 627c3675178848a3a895c3b6797de531 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:14:09.845187 17849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 627c3675178848a3a895c3b6797de531 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 627c3675178848a3a895c3b6797de531. Latest consensus state: current_term: 1 leader_uuid: "627c3675178848a3a895c3b6797de531" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "627c3675178848a3a895c3b6797de531" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44135 } } }
I20260812 11:14:09.845397 17849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 627c3675178848a3a895c3b6797de531 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:09.845855 17846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 627c3675178848a3a895c3b6797de531 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "627c3675178848a3a895c3b6797de531" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "627c3675178848a3a895c3b6797de531" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44135 } } }
I20260812 11:14:09.846036 17846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 627c3675178848a3a895c3b6797de531 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:09.846582 17856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:14:09.850575 17856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:14:09.857188 17856 catalog_manager.cc:1347] Generated new cluster ID: 3188fd9a1ecd425aa0d2d9031732c8d4
I20260812 11:14:09.857282 17856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:14:09.873705 17856 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:14:09.874063 17856 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:14:09.888039 17856 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 627c3675178848a3a895c3b6797de531: Generated new TSK 0
I20260812 11:14:09.888764 17856 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:14:09.896297 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 17723
2026-08-12T11:14:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1726 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:14:09 dist-test-slave-drl0 krb5kdc[17876](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:14:09 dist-test-slave-drl0 krb5kdc[17876](info): set up 2 sockets
Aug 12 11:14:09 dist-test-slave-drl0 krb5kdc[17876](info): commencing operation
krb5kdc: starting...
W20260812 11:14:11.197606 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:14:11 dist-test-slave-drl0 krb5kdc[17876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:14:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:14:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 11:14:11.334635 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38729
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:14:11.457942 17894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:14:11.458294 17894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:14:11.458379 17894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:14:11.462981 17894 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:14:11.463074 17894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:14:11.463114 17894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:14:11.463167 17894 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:14:11.463208 17894 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:14:11.468822 17894 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:14:11.470002 17894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:14:11.471133 17894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:14:11.480058 17901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:11.480427 17903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:11.480443 17906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:11.480504 17894 server_base.cc:1034] running on GCE node
I20260812 11:14:11.481050 17894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:14:11.482129 17894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:14:11.483368 17894 hybrid_clock.cc:648] HybridClock initialized: now 1786533251483342 us; error 35 us; skew 500 ppm
Aug 12 11:14:11 dist-test-slave-drl0 krb5kdc[17876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533251, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:14:11.491160 17894 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 11:14:11.492038 17894 webserver.cc:460] Webserver started at http://127.0.0.1:41749/ using document root <none> and password file <none>
I20260812 11:14:11.492573 17894 fs_manager.cc:359] Metadata directory not provided
I20260812 11:14:11.492631 17894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:14:11.492933 17894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:14:11.495141 17894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "a93ecdbdab544744a8d809df602a92db"
format_stamp: "Formatted at 2026-08-12 11:14:11 on dist-test-slave-drl0"
server_key: "043692863f6183c4c476d0427374939c"
server_key_iv: "fbf3e337eafc83a54f3c288145d13f1c"
server_key_version: "encryptionkey@0"
I20260812 11:14:11.495662 17894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "a93ecdbdab544744a8d809df602a92db"
format_stamp: "Formatted at 2026-08-12 11:14:11 on dist-test-slave-drl0"
server_key: "043692863f6183c4c476d0427374939c"
server_key_iv: "fbf3e337eafc83a54f3c288145d13f1c"
server_key_version: "encryptionkey@0"
I20260812 11:14:11.499310 17894 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 11:14:11.501997 17915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:14:11.503401 17894 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:14:11.503512 17894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "a93ecdbdab544744a8d809df602a92db"
format_stamp: "Formatted at 2026-08-12 11:14:11 on dist-test-slave-drl0"
server_key: "043692863f6183c4c476d0427374939c"
server_key_iv: "fbf3e337eafc83a54f3c288145d13f1c"
server_key_version: "encryptionkey@0"
I20260812 11:14:11.503600 17894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:14:11.531555 17894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:14:11.534829 17894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:14:11.535049 17894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:14:11.544595 17894 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38729
I20260812 11:14:11.544674 18001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38729 every 8 connection(s)
I20260812 11:14:11.545617 17894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:14:11.549407 18002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:14:11.553786 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 17894
I20260812 11:14:11.554013 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:14:11.554512 17587 external_mini_cluster.cc:1428] Setting key 2e1cb8ac154ba9eeee5cfa68595eb9b6
I20260812 11:14:11.555444 18002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a93ecdbdab544744a8d809df602a92db: Bootstrap starting.
I20260812 11:14:11.558306 18002 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a93ecdbdab544744a8d809df602a92db: Neither blocks nor log segments found. Creating new log.
I20260812 11:14:11.559792 18002 log.cc:824] T 00000000000000000000000000000000 P a93ecdbdab544744a8d809df602a92db: Log is configured to *not* fsync() on all Append() calls
I20260812 11:14:11.562058 18002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a93ecdbdab544744a8d809df602a92db: No bootstrap required, opened a new log
Aug 12 11:14:11 dist-test-slave-drl0 krb5kdc[17876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 11:14:11.566627 18002 raft_consensus.cc:348] T 00000000000000000000000000000000 P a93ecdbdab544744a8d809df602a92db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a93ecdbdab544744a8d809df602a92db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38729 } }
I20260812 11:14:11.566984 18002 raft_consensus.cc:374] T 00000000000000000000000000000000 P a93ecdbdab544744a8d809df602a92db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:14:11.567054 18002 raft_consensus.cc:729] T 00000000000000000000000000000000 P a93ecdbdab544744a8d809df602a92db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a93ecdbdab544744a8d809df602a92db, State: Initialized, Role: FOLLOWER
I20260812 11:14:11.567539 18002 consensus_queue.cc:260] T 00000000000000000000000000000000 P a93ecdbdab544744a8d809df602a92db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a93ecdbdab544744a8d809df602a92db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38729 } }
I20260812 11:14:11.567737 18002 raft_consensus.cc:388] T 00000000000000000000000000000000 P a93ecdbdab544744a8d809df602a92db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:14:11.567796 18002 raft_consensus.cc:482] T 00000000000000000000000000000000 P a93ecdbdab544744a8d809df602a92db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:14:11.567909 18002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a93ecdbdab544744a8d809df602a92db [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:14:11.569103 18002 raft_consensus.cc:504] T 00000000000000000000000000000000 P a93ecdbdab544744a8d809df602a92db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a93ecdbdab544744a8d809df602a92db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38729 } }
I20260812 11:14:11.569510 18002 leader_election.cc:302] T 00000000000000000000000000000000 P a93ecdbdab544744a8d809df602a92db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a93ecdbdab544744a8d809df602a92db; no voters:
I20260812 11:14:11.569872 18002 leader_election.cc:288] T 00000000000000000000000000000000 P a93ecdbdab544744a8d809df602a92db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:14:11.570052 18010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a93ecdbdab544744a8d809df602a92db [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:14:11.570294 18010 raft_consensus.cc:686] T 00000000000000000000000000000000 P a93ecdbdab544744a8d809df602a92db [term 1 LEADER]: Becoming Leader. State: Replica: a93ecdbdab544744a8d809df602a92db, State: Running, Role: LEADER
I20260812 11:14:11.570642 18010 consensus_queue.cc:237] T 00000000000000000000000000000000 P a93ecdbdab544744a8d809df602a92db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a93ecdbdab544744a8d809df602a92db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38729 } }
I20260812 11:14:11.571281 18002 sys_catalog.cc:564] T 00000000000000000000000000000000 P a93ecdbdab544744a8d809df602a92db [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:14:11.572350 18014 sys_catalog.cc:455] T 00000000000000000000000000000000 P a93ecdbdab544744a8d809df602a92db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a93ecdbdab544744a8d809df602a92db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a93ecdbdab544744a8d809df602a92db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38729 } } }
I20260812 11:14:11.572517 18014 sys_catalog.cc:458] T 00000000000000000000000000000000 P a93ecdbdab544744a8d809df602a92db [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:11.572643 18013 sys_catalog.cc:455] T 00000000000000000000000000000000 P a93ecdbdab544744a8d809df602a92db [sys.catalog]: SysCatalogTable state changed. Reason: New leader a93ecdbdab544744a8d809df602a92db. Latest consensus state: current_term: 1 leader_uuid: "a93ecdbdab544744a8d809df602a92db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a93ecdbdab544744a8d809df602a92db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38729 } } }
I20260812 11:14:11.572979 18013 sys_catalog.cc:458] T 00000000000000000000000000000000 P a93ecdbdab544744a8d809df602a92db [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:11.573433 18008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:11.556355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.234:45037 (local address 127.0.0.1:38729)
0812 11:14:11.556907 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:11.556938 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:11.556984 (+ 46us) server_negotiation.cc:407] Connection header received
0812 11:14:11.557827 (+ 843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:11.557850 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:11.558302 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:11.558922 (+ 620us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:11.559834 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:11.561233 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:11.562261 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:11.562968 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:11.563139 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:11.566095 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:11.566102 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:11.566119 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:11.566152 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:11.568534 (+ 2382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:11.569111 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:11.569120 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:11.569127 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:11.569318 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:11.569706 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:11.569709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:11.569711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:11.570301 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:11.571172 (+ 871us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:11.571354 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 11:14:11.571583 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":140,"threads_started":1}
I20260812 11:14:11.574893 18018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:14:11.577582 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:14:11.578495 18018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:14:11.584656 18018 catalog_manager.cc:1347] Generated new cluster ID: 6c288824fd434dae939f7d7ab8f1bfb6
I20260812 11:14:11.584915 18018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:14:11.591559 18008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:11.582792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.234:44549 (local address 127.0.0.1:38729)
0812 11:14:11.582903 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:11.582912 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:11.583573 (+ 661us) server_negotiation.cc:407] Connection header received
0812 11:14:11.583723 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:11.583727 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:11.583785 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:11.583871 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:11.584769 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:11.585435 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:11.586297 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:11.586614 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:11.586729 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:11.588344 (+ 1615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:11.588347 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:11.588350 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:11.588382 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:11.589807 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:11.590282 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:11.590287 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:11.590289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:11.590366 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:11.590688 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:11.590691 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:11.590693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:11.590866 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:11.591041 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:11.591307 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 11:14:11.591404 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 11:14:11.605013 18018 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:14:11.606096 18018 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:14:11.628191 18018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a93ecdbdab544744a8d809df602a92db: Generated new TSK 0
I20260812 11:14:11.629227 18018 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:14:11.634894 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 17894
W20260812 11:14:11.649327 17871 connection.cc:535] client connection to 127.0.0.1:38729 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38729): Connection reset by peer (error 104)
2026-08-12T11:14:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1740 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:14:11 dist-test-slave-drl0 krb5kdc[18051](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:14:11 dist-test-slave-drl0 krb5kdc[18051](info): set up 2 sockets
Aug 12 11:14:11 dist-test-slave-drl0 krb5kdc[18051](info): commencing operation
krb5kdc: starting...
W20260812 11:14:12.932387 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:14:13 dist-test-slave-drl0 krb5kdc[18051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:14:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:14:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 11:14:13.071365 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41413
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:14:13.193115 18068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:14:13.193634 18068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:14:13.193804 18068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:14:13.198411 18068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:14:13.198472 18068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:14:13.198493 18068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:14:13.198509 18068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:14:13.198524 18068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:14:13.206514 18068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:14:13.207660 18068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:14:13.208674 18068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:14:13.217664 18075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:13.217643 18081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:13.217729 18076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:13.218183 18068 server_base.cc:1034] running on GCE node
I20260812 11:14:13.218730 18068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:14:13.219848 18068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:14:13.221156 18068 hybrid_clock.cc:648] HybridClock initialized: now 1786533253221136 us; error 42 us; skew 500 ppm
Aug 12 11:14:13 dist-test-slave-drl0 krb5kdc[18051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533253, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:14:13.224495 18068 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 11:14:13.225338 18068 webserver.cc:460] Webserver started at http://127.0.0.1:39215/ using document root <none> and password file <none>
I20260812 11:14:13.225865 18068 fs_manager.cc:359] Metadata directory not provided
I20260812 11:14:13.225924 18068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:14:13.226085 18068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:14:13.228307 18068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "15e44b2ae91f4ce79259b1e2a8929c55"
format_stamp: "Formatted at 2026-08-12 11:14:13 on dist-test-slave-drl0"
server_key: "ad449057de703ad9ab262bbc8760b929"
server_key_iv: "31dba17f42655036a6e467d6207c1dcf"
server_key_version: "encryptionkey@0"
I20260812 11:14:13.228881 18068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "15e44b2ae91f4ce79259b1e2a8929c55"
format_stamp: "Formatted at 2026-08-12 11:14:13 on dist-test-slave-drl0"
server_key: "ad449057de703ad9ab262bbc8760b929"
server_key_iv: "31dba17f42655036a6e467d6207c1dcf"
server_key_version: "encryptionkey@0"
I20260812 11:14:13.232779 18068 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 11:14:13.236004 18091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:14:13.237565 18068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:14:13.237686 18068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "15e44b2ae91f4ce79259b1e2a8929c55"
format_stamp: "Formatted at 2026-08-12 11:14:13 on dist-test-slave-drl0"
server_key: "ad449057de703ad9ab262bbc8760b929"
server_key_iv: "31dba17f42655036a6e467d6207c1dcf"
server_key_version: "encryptionkey@0"
I20260812 11:14:13.237780 18068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:14:13.254405 18068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:14:13.257865 18068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:14:13.258049 18068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:14:13.265926 18068 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41413
I20260812 11:14:13.265946 18175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41413 every 8 connection(s)
I20260812 11:14:13.266880 18068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:14:13.269970 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 18068
I20260812 11:14:13.270151 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:14:13.270521 17587 external_mini_cluster.cc:1428] Setting key 876eba7df45a10f3810c0196ad4a9303
I20260812 11:14:13.270701 18176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:14:13.277091 18176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15e44b2ae91f4ce79259b1e2a8929c55: Bootstrap starting.
I20260812 11:14:13.279577 18176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15e44b2ae91f4ce79259b1e2a8929c55: Neither blocks nor log segments found. Creating new log.
I20260812 11:14:13.279876 18180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:13.272026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.234:50483 (local address 127.0.0.1:41413)
0812 11:14:13.272545 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:13.272561 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:13.272595 (+ 34us) server_negotiation.cc:407] Connection header received
0812 11:14:13.273498 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:13.273514 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:13.273841 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:13.274177 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:13.274589 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:13.275653 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:13.276523 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:13.276979 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:13.277141 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:13.278125 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:13.278129 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:13.278140 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:14:13.278536 (+ 396us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:13.278596 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:14:13.278621 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:13.278763 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:13.278982 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 11:14:13.279129 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":172,"threads_started":1}
I20260812 11:14:13.280480 18176 log.cc:824] T 00000000000000000000000000000000 P 15e44b2ae91f4ce79259b1e2a8929c55: Log is configured to *not* fsync() on all Append() calls
I20260812 11:14:13.283036 18176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15e44b2ae91f4ce79259b1e2a8929c55: No bootstrap required, opened a new log
I20260812 11:14:13.283160 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:14:13.286662 18176 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15e44b2ae91f4ce79259b1e2a8929c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15e44b2ae91f4ce79259b1e2a8929c55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41413 } }
I20260812 11:14:13.286967 18176 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15e44b2ae91f4ce79259b1e2a8929c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:14:13.287029 18176 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15e44b2ae91f4ce79259b1e2a8929c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15e44b2ae91f4ce79259b1e2a8929c55, State: Initialized, Role: FOLLOWER
I20260812 11:14:13.287509 18176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15e44b2ae91f4ce79259b1e2a8929c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15e44b2ae91f4ce79259b1e2a8929c55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41413 } }
I20260812 11:14:13.287681 18176 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15e44b2ae91f4ce79259b1e2a8929c55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:14:13.287743 18176 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15e44b2ae91f4ce79259b1e2a8929c55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:14:13.287876 18176 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15e44b2ae91f4ce79259b1e2a8929c55 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:14:13.288947 18176 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15e44b2ae91f4ce79259b1e2a8929c55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15e44b2ae91f4ce79259b1e2a8929c55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41413 } }
I20260812 11:14:13.289328 18176 leader_election.cc:302] T 00000000000000000000000000000000 P 15e44b2ae91f4ce79259b1e2a8929c55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15e44b2ae91f4ce79259b1e2a8929c55; no voters:
I20260812 11:14:13.289669 18176 leader_election.cc:288] T 00000000000000000000000000000000 P 15e44b2ae91f4ce79259b1e2a8929c55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:14:13.289788 18198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15e44b2ae91f4ce79259b1e2a8929c55 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:14:13.290035 18198 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15e44b2ae91f4ce79259b1e2a8929c55 [term 1 LEADER]: Becoming Leader. State: Replica: 15e44b2ae91f4ce79259b1e2a8929c55, State: Running, Role: LEADER
I20260812 11:14:13.290355 18180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:13.285522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.234:48777 (local address 127.0.0.1:41413)
0812 11:14:13.285635 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:13.285638 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:13.285650 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:14:13.285679 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:13.285681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:13.285713 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:13.285776 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:13.286101 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:13.286681 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:13.287847 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:13.288304 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:13.288398 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:13.289667 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:13.289671 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:13.289673 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:14:13.289690 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:13.289720 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:14:13.289734 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:13.289837 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:13.290196 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 11:14:13.290231 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 11:14:13.290391 18198 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15e44b2ae91f4ce79259b1e2a8929c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15e44b2ae91f4ce79259b1e2a8929c55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41413 } }
I20260812 11:14:13.291004 18176 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15e44b2ae91f4ce79259b1e2a8929c55 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:14:13.292652 18201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15e44b2ae91f4ce79259b1e2a8929c55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15e44b2ae91f4ce79259b1e2a8929c55. Latest consensus state: current_term: 1 leader_uuid: "15e44b2ae91f4ce79259b1e2a8929c55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15e44b2ae91f4ce79259b1e2a8929c55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41413 } } }
I20260812 11:14:13.292701 18200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15e44b2ae91f4ce79259b1e2a8929c55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15e44b2ae91f4ce79259b1e2a8929c55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15e44b2ae91f4ce79259b1e2a8929c55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41413 } } }
I20260812 11:14:13.292821 18201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15e44b2ae91f4ce79259b1e2a8929c55 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:13.292819 18200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15e44b2ae91f4ce79259b1e2a8929c55 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:13.293227 18214 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:14:13.296144 18214 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:14:13.301869 18214 catalog_manager.cc:1347] Generated new cluster ID: 12488e48d0e8472880977fa18d5225bd
I20260812 11:14:13.301941 18214 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:14:13.328819 18214 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:14:13.329931 18214 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:14:13.345734 18214 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15e44b2ae91f4ce79259b1e2a8929c55: Generated new TSK 0
I20260812 11:14:13.347088 18214 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:14:13.363395 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 18068
W20260812 11:14:13.379777 18048 connection.cc:535] client connection to 127.0.0.1:41413 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41413): Connection reset by peer (error 104)
2026-08-12T11:14:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1730 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:14:13 dist-test-slave-drl0 krb5kdc[18233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:14:13 dist-test-slave-drl0 krb5kdc[18233](info): set up 2 sockets
Aug 12 11:14:13 dist-test-slave-drl0 krb5kdc[18233](info): commencing operation
krb5kdc: starting...
W20260812 11:14:14.663535 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:14:14 dist-test-slave-drl0 krb5kdc[18233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:14:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:14:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 11:14:14.799309 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33033
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:14:14.922804 18258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:14:14.923218 18258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:14:14.923295 18258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:14:14.930017 18258 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:14:14.930147 18258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:14:14.930191 18258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:14:14.930229 18258 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:14:14.930269 18258 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:14:14.939380 18258 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:14:14.940831 18258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:14:14.941982 18258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:14:14.951268 18267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:14.951390 18270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:14.951268 18274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:14.951514 18258 server_base.cc:1034] running on GCE node
I20260812 11:14:14.952237 18258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:14:14.953365 18258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:14:14.954576 18258 hybrid_clock.cc:648] HybridClock initialized: now 1786533254954548 us; error 40 us; skew 500 ppm
Aug 12 11:14:14 dist-test-slave-drl0 krb5kdc[18233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533254, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:14:14.958844 18258 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 11:14:14.960330 18258 webserver.cc:460] Webserver started at http://127.0.0.1:44637/ using document root <none> and password file <none>
I20260812 11:14:14.960987 18258 fs_manager.cc:359] Metadata directory not provided
I20260812 11:14:14.961086 18258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:14:14.961328 18258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:14:14.963565 18258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "1edac762077b49ae97b5fe9f8bc0c239"
format_stamp: "Formatted at 2026-08-12 11:14:14 on dist-test-slave-drl0"
server_key: "97de6fa851b7188f4d1becfc310cf846"
server_key_iv: "196a3f41beb7fbd8d9bf9c9f4b276db2"
server_key_version: "encryptionkey@0"
I20260812 11:14:14.964350 18258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "1edac762077b49ae97b5fe9f8bc0c239"
format_stamp: "Formatted at 2026-08-12 11:14:14 on dist-test-slave-drl0"
server_key: "97de6fa851b7188f4d1becfc310cf846"
server_key_iv: "196a3f41beb7fbd8d9bf9c9f4b276db2"
server_key_version: "encryptionkey@0"
I20260812 11:14:14.968668 18258 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 11:14:14.971794 18284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:14:14.973379 18258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:14:14.973544 18258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "1edac762077b49ae97b5fe9f8bc0c239"
format_stamp: "Formatted at 2026-08-12 11:14:14 on dist-test-slave-drl0"
server_key: "97de6fa851b7188f4d1becfc310cf846"
server_key_iv: "196a3f41beb7fbd8d9bf9c9f4b276db2"
server_key_version: "encryptionkey@0"
I20260812 11:14:14.973668 18258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:14:14.978509 18258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:14:14.982105 18258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:14:14.982362 18258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:14:14.992206 18258 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33033
I20260812 11:14:14.992237 18369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33033 every 8 connection(s)
I20260812 11:14:14.993414 18258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:14:14.996511 18374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:14:14.998286 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 18258
I20260812 11:14:14.998445 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:14:14.998782 17587 external_mini_cluster.cc:1428] Setting key bdf445827b9d32a56731c6d61b26d26c
I20260812 11:14:15.003120 18374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1edac762077b49ae97b5fe9f8bc0c239: Bootstrap starting.
I20260812 11:14:15.005730 18374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1edac762077b49ae97b5fe9f8bc0c239: Neither blocks nor log segments found. Creating new log.
I20260812 11:14:15.005992 18378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:15.000503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.234:36667 (local address 127.0.0.1:33033)
0812 11:14:15.001155 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:15.001179 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:15.001239 (+ 60us) server_negotiation.cc:407] Connection header received
0812 11:14:15.002207 (+ 968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:15.002231 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:15.002803 (+ 572us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:15.003491 (+ 688us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:15.004648 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:15.004654 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:15.004692 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:14:15.004711 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:15.004818 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:14:15.004850 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:15.005007 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:15.005085 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 11:14:15.005269 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":168,"threads_started":1}
I20260812 11:14:15.006567 18374 log.cc:824] T 00000000000000000000000000000000 P 1edac762077b49ae97b5fe9f8bc0c239: Log is configured to *not* fsync() on all Append() calls
I20260812 11:14:15.008808 18374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1edac762077b49ae97b5fe9f8bc0c239: No bootstrap required, opened a new log
I20260812 11:14:15.009367 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:14:15.012795 18374 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1edac762077b49ae97b5fe9f8bc0c239 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1edac762077b49ae97b5fe9f8bc0c239" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33033 } }
I20260812 11:14:15.013095 18374 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1edac762077b49ae97b5fe9f8bc0c239 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:14:15.013185 18374 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1edac762077b49ae97b5fe9f8bc0c239 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1edac762077b49ae97b5fe9f8bc0c239, State: Initialized, Role: FOLLOWER
I20260812 11:14:15.013337 18378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:15.011490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.234:52259 (local address 127.0.0.1:33033)
0812 11:14:15.011638 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:15.011643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:15.011738 (+ 95us) server_negotiation.cc:407] Connection header received
0812 11:14:15.011838 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:15.011843 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:15.011891 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:15.011996 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:15.012791 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:15.012794 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:15.012797 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:14:15.012812 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:15.012843 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:14:15.012856 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:15.012964 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:15.013017 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 11:14:15.013060 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 11:14:15.013692 18374 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1edac762077b49ae97b5fe9f8bc0c239 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1edac762077b49ae97b5fe9f8bc0c239" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33033 } }
I20260812 11:14:15.013841 18374 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1edac762077b49ae97b5fe9f8bc0c239 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:14:15.013922 18374 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1edac762077b49ae97b5fe9f8bc0c239 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:14:15.014040 18374 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1edac762077b49ae97b5fe9f8bc0c239 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:14:15.015074 18374 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1edac762077b49ae97b5fe9f8bc0c239 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1edac762077b49ae97b5fe9f8bc0c239" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33033 } }
I20260812 11:14:15.015408 18374 leader_election.cc:302] T 00000000000000000000000000000000 P 1edac762077b49ae97b5fe9f8bc0c239 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1edac762077b49ae97b5fe9f8bc0c239; no voters:
I20260812 11:14:15.015811 18374 leader_election.cc:288] T 00000000000000000000000000000000 P 1edac762077b49ae97b5fe9f8bc0c239 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:14:15.015851 18389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1edac762077b49ae97b5fe9f8bc0c239 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:14:15.016110 18389 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1edac762077b49ae97b5fe9f8bc0c239 [term 1 LEADER]: Becoming Leader. State: Replica: 1edac762077b49ae97b5fe9f8bc0c239, State: Running, Role: LEADER
I20260812 11:14:15.016420 18389 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1edac762077b49ae97b5fe9f8bc0c239 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1edac762077b49ae97b5fe9f8bc0c239" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33033 } }
I20260812 11:14:15.017048 18374 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1edac762077b49ae97b5fe9f8bc0c239 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:14:15.018059 18390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1edac762077b49ae97b5fe9f8bc0c239 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1edac762077b49ae97b5fe9f8bc0c239. Latest consensus state: current_term: 1 leader_uuid: "1edac762077b49ae97b5fe9f8bc0c239" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1edac762077b49ae97b5fe9f8bc0c239" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33033 } } }
I20260812 11:14:15.018191 18390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1edac762077b49ae97b5fe9f8bc0c239 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:15.018515 18391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1edac762077b49ae97b5fe9f8bc0c239 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1edac762077b49ae97b5fe9f8bc0c239" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1edac762077b49ae97b5fe9f8bc0c239" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33033 } } }
I20260812 11:14:15.018610 18391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1edac762077b49ae97b5fe9f8bc0c239 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:15.019553 18407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:14:15.022369 18407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:14:15.027878 18407 catalog_manager.cc:1347] Generated new cluster ID: eeea8b23f6b045689262c55f50499918
I20260812 11:14:15.027943 18407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:14:15.049208 18407 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:14:15.049546 18407 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:14:15.054905 18407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1edac762077b49ae97b5fe9f8bc0c239: Generated new TSK 0
I20260812 11:14:15.055646 18407 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:14:15.069365 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 18258
2026-08-12T11:14:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1703 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:14:15 dist-test-slave-drl0 krb5kdc[18430](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:14:15 dist-test-slave-drl0 krb5kdc[18430](info): set up 2 sockets
Aug 12 11:14:15 dist-test-slave-drl0 krb5kdc[18430](info): commencing operation
krb5kdc: starting...
W20260812 11:14:16.372375 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:14:16 dist-test-slave-drl0 krb5kdc[18430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:14:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:14:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 11:14:16.506335 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46713
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:14:16.634506 18461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:14:16.634909 18461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:14:16.635139 18461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:14:16.640365 18461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:14:16.640466 18461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:14:16.640506 18461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:14:16.640563 18461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:14:16.640602 18461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:14:16.647121 18461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46713
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:14:16.648263 18461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:14:16.649307 18461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:14:16.658548 18476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:16.658565 18473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:16.658860 18461 server_base.cc:1034] running on GCE node
W20260812 11:14:16.658586 18471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:16.659564 18461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:14:16.660629 18461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:14:16.661866 18461 hybrid_clock.cc:648] HybridClock initialized: now 1786533256661815 us; error 64 us; skew 500 ppm
Aug 12 11:14:16 dist-test-slave-drl0 krb5kdc[18430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533256, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:14:16.667748 18461 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 11:14:16.668695 18461 webserver.cc:460] Webserver started at http://127.0.0.1:45077/ using document root <none> and password file <none>
I20260812 11:14:16.669232 18461 fs_manager.cc:359] Metadata directory not provided
I20260812 11:14:16.669301 18461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:14:16.669466 18461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:14:16.671518 18461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "919f9ff5b4434fef9a198a15d73320e6"
format_stamp: "Formatted at 2026-08-12 11:14:16 on dist-test-slave-drl0"
server_key: "e4fdc90fc0a01c083e84b599729ab99c"
server_key_iv: "4c7b7307ccee3b0f4ac14536f8bdd881"
server_key_version: "encryptionkey@0"
I20260812 11:14:16.672029 18461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "919f9ff5b4434fef9a198a15d73320e6"
format_stamp: "Formatted at 2026-08-12 11:14:16 on dist-test-slave-drl0"
server_key: "e4fdc90fc0a01c083e84b599729ab99c"
server_key_iv: "4c7b7307ccee3b0f4ac14536f8bdd881"
server_key_version: "encryptionkey@0"
I20260812 11:14:16.675763 18461 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:14:16.678143 18490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:14:16.679483 18461 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:14:16.679589 18461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "919f9ff5b4434fef9a198a15d73320e6"
format_stamp: "Formatted at 2026-08-12 11:14:16 on dist-test-slave-drl0"
server_key: "e4fdc90fc0a01c083e84b599729ab99c"
server_key_iv: "4c7b7307ccee3b0f4ac14536f8bdd881"
server_key_version: "encryptionkey@0"
I20260812 11:14:16.679662 18461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:14:16.693351 18461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:14:16.697475 18461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:14:16.697705 18461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:14:16.707211 18461 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46713
I20260812 11:14:16.707279 18568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46713 every 8 connection(s)
I20260812 11:14:16.708241 18461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:14:16.711213 18569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:14:16.713631 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 18461
I20260812 11:14:16.713792 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:14:16.714094 17587 external_mini_cluster.cc:1428] Setting key ced7e325ea8a362214ae9fb358b093b6
I20260812 11:14:16.718376 18569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 919f9ff5b4434fef9a198a15d73320e6: Bootstrap starting.
I20260812 11:14:16.720999 18569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 919f9ff5b4434fef9a198a15d73320e6: Neither blocks nor log segments found. Creating new log.
I20260812 11:14:16.721921 18569 log.cc:824] T 00000000000000000000000000000000 P 919f9ff5b4434fef9a198a15d73320e6: Log is configured to *not* fsync() on all Append() calls
Aug 12 11:14:16 dist-test-slave-drl0 krb5kdc[18430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 11:14:16.724572 18569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 919f9ff5b4434fef9a198a15d73320e6: No bootstrap required, opened a new log
I20260812 11:14:16.728353 18569 raft_consensus.cc:348] T 00000000000000000000000000000000 P 919f9ff5b4434fef9a198a15d73320e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "919f9ff5b4434fef9a198a15d73320e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46713 } }
I20260812 11:14:16.728605 18569 raft_consensus.cc:374] T 00000000000000000000000000000000 P 919f9ff5b4434fef9a198a15d73320e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:14:16.728662 18569 raft_consensus.cc:729] T 00000000000000000000000000000000 P 919f9ff5b4434fef9a198a15d73320e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 919f9ff5b4434fef9a198a15d73320e6, State: Initialized, Role: FOLLOWER
I20260812 11:14:16.729127 18569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 919f9ff5b4434fef9a198a15d73320e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "919f9ff5b4434fef9a198a15d73320e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46713 } }
I20260812 11:14:16.729287 18569 raft_consensus.cc:388] T 00000000000000000000000000000000 P 919f9ff5b4434fef9a198a15d73320e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:14:16.729372 18569 raft_consensus.cc:482] T 00000000000000000000000000000000 P 919f9ff5b4434fef9a198a15d73320e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:14:16.729487 18569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 919f9ff5b4434fef9a198a15d73320e6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:14:16.729638 18575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:16.715533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.234:38007 (local address 127.0.0.1:46713)
0812 11:14:16.715938 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:16.715955 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:16.715989 (+ 34us) server_negotiation.cc:407] Connection header received
0812 11:14:16.716675 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:16.716690 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:16.717033 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:16.717316 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:16.718598 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:16.719672 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:16.720816 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:16.721353 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:16.721480 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:16.724397 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:16.724403 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:16.724416 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:16.724446 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:16.726306 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:16.726879 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:16.726884 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:16.726889 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:16.727071 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:16.727489 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:16.727493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:16.727495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:16.727907 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:16.728156 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:16.728495 (+ 339us) server_negotiation.cc:299] Negotiation successful
0812 11:14:16.728728 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":128,"threads_started":1}
I20260812 11:14:16.730470 18569 raft_consensus.cc:504] T 00000000000000000000000000000000 P 919f9ff5b4434fef9a198a15d73320e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "919f9ff5b4434fef9a198a15d73320e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46713 } }
I20260812 11:14:16.730832 18569 leader_election.cc:302] T 00000000000000000000000000000000 P 919f9ff5b4434fef9a198a15d73320e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 919f9ff5b4434fef9a198a15d73320e6; no voters:
I20260812 11:14:16.731182 18569 leader_election.cc:288] T 00000000000000000000000000000000 P 919f9ff5b4434fef9a198a15d73320e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:14:16.731345 18577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 919f9ff5b4434fef9a198a15d73320e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:14:16.731678 18577 raft_consensus.cc:686] T 00000000000000000000000000000000 P 919f9ff5b4434fef9a198a15d73320e6 [term 1 LEADER]: Becoming Leader. State: Replica: 919f9ff5b4434fef9a198a15d73320e6, State: Running, Role: LEADER
I20260812 11:14:16.732045 18577 consensus_queue.cc:237] T 00000000000000000000000000000000 P 919f9ff5b4434fef9a198a15d73320e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "919f9ff5b4434fef9a198a15d73320e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46713 } }
I20260812 11:14:16.732398 18569 sys_catalog.cc:564] T 00000000000000000000000000000000 P 919f9ff5b4434fef9a198a15d73320e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:14:16.732924 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:14:16.733786 18579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919f9ff5b4434fef9a198a15d73320e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "919f9ff5b4434fef9a198a15d73320e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "919f9ff5b4434fef9a198a15d73320e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46713 } } }
I20260812 11:14:16.733922 18579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919f9ff5b4434fef9a198a15d73320e6 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:16.734306 18581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919f9ff5b4434fef9a198a15d73320e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 919f9ff5b4434fef9a198a15d73320e6. Latest consensus state: current_term: 1 leader_uuid: "919f9ff5b4434fef9a198a15d73320e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "919f9ff5b4434fef9a198a15d73320e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46713 } } }
I20260812 11:14:16.734407 18581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919f9ff5b4434fef9a198a15d73320e6 [sys.catalog]: This master's current role is: LEADER
W20260812 11:14:16.738613 18606 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 919f9ff5b4434fef9a198a15d73320e6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:14:16.738761 18606 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:14:16.738924 18594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:14:16.740000 18594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:14:16.744357 18575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:16.735758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.234:55643 (local address 127.0.0.1:46713)
0812 11:14:16.735895 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:16.735900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:16.736162 (+ 262us) server_negotiation.cc:407] Connection header received
0812 11:14:16.736222 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:16.736225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:16.736280 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:16.736384 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:16.737053 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:16.737571 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:16.738383 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:16.738614 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:16.738718 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:16.740446 (+ 1728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:16.740451 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:16.740454 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:16.740483 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:16.742258 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:16.742818 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:16.742823 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:16.742826 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:16.742965 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:16.743383 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:16.743388 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:16.743392 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:16.743607 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:16.743760 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:16.744076 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 11:14:16.744174 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:14:16.751060 18594 catalog_manager.cc:1347] Generated new cluster ID: b371494568734b8886e1629a47bbd47f
I20260812 11:14:16.751125 18594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:14:16.770586 18594 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:14:16.771601 18594 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:14:16.779955 18594 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 919f9ff5b4434fef9a198a15d73320e6: Generated new TSK 0
I20260812 11:14:16.780635 18594 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:14:16.789757 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 18461
W20260812 11:14:16.809266 18426 connection.cc:535] client connection to 127.0.0.1:46713 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46713): Connection reset by peer (error 104)
2026-08-12T11:14:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1725 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:14:16 dist-test-slave-drl0 krb5kdc[18618](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:14:16 dist-test-slave-drl0 krb5kdc[18618](info): set up 2 sockets
Aug 12 11:14:16 dist-test-slave-drl0 krb5kdc[18618](info): commencing operation
krb5kdc: starting...
W20260812 11:14:18.095330 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:14:18 dist-test-slave-drl0 krb5kdc[18618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:14:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:14:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 11:14:18.231495 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36985
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36985
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:14:18.353956 18635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:14:18.354415 18635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:14:18.354508 18635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:14:18.360309 18635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:14:18.360441 18635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:14:18.360479 18635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:14:18.360543 18635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:14:18.360605 18635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:14:18.367015 18635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36985
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:14:18.368155 18635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:14:18.369325 18635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:14:18.378547 18648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:18.378651 18644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:18.378654 18645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:18.378852 18635 server_base.cc:1034] running on GCE node
I20260812 11:14:18.379699 18635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:14:18.380815 18635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:14:18.382086 18635 hybrid_clock.cc:648] HybridClock initialized: now 1786533258382063 us; error 35 us; skew 500 ppm
Aug 12 11:14:18 dist-test-slave-drl0 krb5kdc[18618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533258, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:14:18.385326 18635 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 11:14:18.386168 18635 webserver.cc:460] Webserver started at http://127.0.0.1:38677/ using document root <none> and password file <none>
I20260812 11:14:18.386742 18635 fs_manager.cc:359] Metadata directory not provided
I20260812 11:14:18.386807 18635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:14:18.387061 18635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:14:18.389108 18635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "2bf62af0dc394917af13c99e9238b6d2"
format_stamp: "Formatted at 2026-08-12 11:14:18 on dist-test-slave-drl0"
server_key: "b6df9b231363d7640a278bcfd4b59c91"
server_key_iv: "a3335b3cd30367128a86e08e9a674a29"
server_key_version: "encryptionkey@0"
I20260812 11:14:18.389680 18635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "2bf62af0dc394917af13c99e9238b6d2"
format_stamp: "Formatted at 2026-08-12 11:14:18 on dist-test-slave-drl0"
server_key: "b6df9b231363d7640a278bcfd4b59c91"
server_key_iv: "a3335b3cd30367128a86e08e9a674a29"
server_key_version: "encryptionkey@0"
I20260812 11:14:18.393321 18635 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260812 11:14:18.395838 18664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:14:18.397084 18635 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 11:14:18.397192 18635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "2bf62af0dc394917af13c99e9238b6d2"
format_stamp: "Formatted at 2026-08-12 11:14:18 on dist-test-slave-drl0"
server_key: "b6df9b231363d7640a278bcfd4b59c91"
server_key_iv: "a3335b3cd30367128a86e08e9a674a29"
server_key_version: "encryptionkey@0"
I20260812 11:14:18.397265 18635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:14:18.412767 18635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:14:18.417259 18635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:14:18.417425 18635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:14:18.426335 18635 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36985
I20260812 11:14:18.426409 18735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36985 every 8 connection(s)
I20260812 11:14:18.427409 18635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:14:18.430222 18736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:14:18.430423 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 18635
I20260812 11:14:18.430574 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:14:18.430884 17587 external_mini_cluster.cc:1428] Setting key 9cf5b1093949fd4e200da1e5fe9fb6bb
I20260812 11:14:18.437485 18736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bf62af0dc394917af13c99e9238b6d2: Bootstrap starting.
I20260812 11:14:18.440183 18736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2bf62af0dc394917af13c99e9238b6d2: Neither blocks nor log segments found. Creating new log.
I20260812 11:14:18.441069 18736 log.cc:824] T 00000000000000000000000000000000 P 2bf62af0dc394917af13c99e9238b6d2: Log is configured to *not* fsync() on all Append() calls
I20260812 11:14:18.441861 18741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:18.432402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.234:41555 (local address 127.0.0.1:36985)
0812 11:14:18.432851 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:18.432870 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:18.432914 (+ 44us) server_negotiation.cc:407] Connection header received
0812 11:14:18.433703 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:18.433727 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:18.434168 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:18.434525 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:18.434967 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:18.436471 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:18.437389 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:18.438041 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:18.438203 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:18.439499 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:18.439506 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:18.439521 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:14:18.440009 (+ 488us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:18.440088 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:14:18.440124 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:18.440278 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:18.440639 (+ 361us) server_negotiation.cc:299] Negotiation successful
0812 11:14:18.440766 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":126,"threads_started":1}
I20260812 11:14:18.444077 18736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bf62af0dc394917af13c99e9238b6d2: No bootstrap required, opened a new log
I20260812 11:14:18.445869 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:14:18.448702 18736 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bf62af0dc394917af13c99e9238b6d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bf62af0dc394917af13c99e9238b6d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36985 } }
I20260812 11:14:18.448976 18736 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2bf62af0dc394917af13c99e9238b6d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:14:18.449087 18736 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bf62af0dc394917af13c99e9238b6d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bf62af0dc394917af13c99e9238b6d2, State: Initialized, Role: FOLLOWER
I20260812 11:14:18.449596 18736 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bf62af0dc394917af13c99e9238b6d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bf62af0dc394917af13c99e9238b6d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36985 } }
I20260812 11:14:18.449730 18736 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2bf62af0dc394917af13c99e9238b6d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:14:18.449817 18736 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2bf62af0dc394917af13c99e9238b6d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:14:18.449914 18736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bf62af0dc394917af13c99e9238b6d2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:14:18.451040 18736 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2bf62af0dc394917af13c99e9238b6d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bf62af0dc394917af13c99e9238b6d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36985 } }
I20260812 11:14:18.451396 18736 leader_election.cc:302] T 00000000000000000000000000000000 P 2bf62af0dc394917af13c99e9238b6d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2bf62af0dc394917af13c99e9238b6d2; no voters:
I20260812 11:14:18.451740 18736 leader_election.cc:288] T 00000000000000000000000000000000 P 2bf62af0dc394917af13c99e9238b6d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:14:18.451804 18749 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2bf62af0dc394917af13c99e9238b6d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:14:18.452091 18749 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2bf62af0dc394917af13c99e9238b6d2 [term 1 LEADER]: Becoming Leader. State: Replica: 2bf62af0dc394917af13c99e9238b6d2, State: Running, Role: LEADER
I20260812 11:14:18.452452 18749 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bf62af0dc394917af13c99e9238b6d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bf62af0dc394917af13c99e9238b6d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36985 } }
I20260812 11:14:18.453084 18736 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bf62af0dc394917af13c99e9238b6d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:14:18.453194 18741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:18.448405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.234:40631 (local address 127.0.0.1:36985)
0812 11:14:18.448512 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:18.448516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:18.448699 (+ 183us) server_negotiation.cc:407] Connection header received
0812 11:14:18.448869 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:18.448871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:18.448914 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:18.449033 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:18.449354 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:18.450011 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:18.451204 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:18.451501 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:18.451619 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:18.452547 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:18.452553 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:18.452556 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:14:18.452571 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:18.452597 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:14:18.452610 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:18.452696 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:18.452999 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 11:14:18.453031 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 11:14:18.454378 18752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bf62af0dc394917af13c99e9238b6d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2bf62af0dc394917af13c99e9238b6d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bf62af0dc394917af13c99e9238b6d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36985 } } }
I20260812 11:14:18.454391 18751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bf62af0dc394917af13c99e9238b6d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bf62af0dc394917af13c99e9238b6d2. Latest consensus state: current_term: 1 leader_uuid: "2bf62af0dc394917af13c99e9238b6d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bf62af0dc394917af13c99e9238b6d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36985 } } }
I20260812 11:14:18.454536 18752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bf62af0dc394917af13c99e9238b6d2 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:18.454538 18751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bf62af0dc394917af13c99e9238b6d2 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:18.455053 18767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:14:18.458642 18767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:14:18.464468 18767 catalog_manager.cc:1347] Generated new cluster ID: 94a5be6983a84e3682cd59bc10b2f51a
I20260812 11:14:18.464547 18767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:14:18.481258 18767 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:14:18.482563 18767 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:14:18.489130 18767 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2bf62af0dc394917af13c99e9238b6d2: Generated new TSK 0
I20260812 11:14:18.489838 18767 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:14:18.500344 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 18635
W20260812 11:14:18.519418 18616 connection.cc:535] client connection to 127.0.0.1:36985 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36985): Connection reset by peer (error 104)
2026-08-12T11:14:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1710 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:14:18 dist-test-slave-drl0 krb5kdc[18783](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:14:18 dist-test-slave-drl0 krb5kdc[18783](info): set up 2 sockets
Aug 12 11:14:18 dist-test-slave-drl0 krb5kdc[18783](info): commencing operation
krb5kdc: starting...
W20260812 11:14:19.804677 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:14:19 dist-test-slave-drl0 krb5kdc[18783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:14:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:14:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 11:14:19.945917 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45795
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:14:20.062252 18805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:14:20.062517 18805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:14:20.062558 18805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:14:20.066620 18805 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:14:20.066686 18805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:14:20.066706 18805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:14:20.066722 18805 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:14:20.066738 18805 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:14:20.071938 18805 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45795
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:14:20.072887 18805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:14:20.073975 18805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:14:20.083366 18815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:20.083501 18814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:20.083410 18819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:20.084190 18805 server_base.cc:1034] running on GCE node
I20260812 11:14:20.085253 18805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:14:20.086359 18805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:14:20.087633 18805 hybrid_clock.cc:648] HybridClock initialized: now 1786533260087584 us; error 48 us; skew 500 ppm
Aug 12 11:14:20 dist-test-slave-drl0 krb5kdc[18783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533260, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:14:20.091657 18805 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 11:14:20.092631 18805 webserver.cc:460] Webserver started at http://127.0.0.1:35435/ using document root <none> and password file <none>
I20260812 11:14:20.093221 18805 fs_manager.cc:359] Metadata directory not provided
I20260812 11:14:20.093320 18805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:14:20.093670 18805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:14:20.096004 18805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "f302a3d23eb84c8d92ca371d7ff27657"
format_stamp: "Formatted at 2026-08-12 11:14:20 on dist-test-slave-drl0"
server_key: "648c57a3692c6e737ba1c3ca23199bda"
server_key_iv: "0182792f7b876a8152888d4fdad656a8"
server_key_version: "encryptionkey@0"
I20260812 11:14:20.096587 18805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "f302a3d23eb84c8d92ca371d7ff27657"
format_stamp: "Formatted at 2026-08-12 11:14:20 on dist-test-slave-drl0"
server_key: "648c57a3692c6e737ba1c3ca23199bda"
server_key_iv: "0182792f7b876a8152888d4fdad656a8"
server_key_version: "encryptionkey@0"
I20260812 11:14:20.100350 18805 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:14:20.102888 18832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:14:20.104766 18805 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:14:20.105026 18805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "f302a3d23eb84c8d92ca371d7ff27657"
format_stamp: "Formatted at 2026-08-12 11:14:20 on dist-test-slave-drl0"
server_key: "648c57a3692c6e737ba1c3ca23199bda"
server_key_iv: "0182792f7b876a8152888d4fdad656a8"
server_key_version: "encryptionkey@0"
I20260812 11:14:20.105196 18805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:14:20.110616 18805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:14:20.114499 18805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:14:20.114734 18805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:14:20.123102 18805 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45795
I20260812 11:14:20.124243 18805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:14:20.125356 18914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45795 every 8 connection(s)
I20260812 11:14:20.127375 18918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:14:20.132797 18918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f302a3d23eb84c8d92ca371d7ff27657: Bootstrap starting.
I20260812 11:14:20.133682 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 18805
I20260812 11:14:20.133843 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:14:20.134158 17587 external_mini_cluster.cc:1428] Setting key 4ea67d8943064459518be9e00933b1f0
I20260812 11:14:20.135946 18918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f302a3d23eb84c8d92ca371d7ff27657: Neither blocks nor log segments found. Creating new log.
I20260812 11:14:20.136729 18918 log.cc:824] T 00000000000000000000000000000000 P f302a3d23eb84c8d92ca371d7ff27657: Log is configured to *not* fsync() on all Append() calls
I20260812 11:14:20.139036 18918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f302a3d23eb84c8d92ca371d7ff27657: No bootstrap required, opened a new log
I20260812 11:14:20.139509 18921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:20.135375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.234:50321 (local address 127.0.0.1:45795)
0812 11:14:20.135897 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:20.135913 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:20.135971 (+ 58us) server_negotiation.cc:407] Connection header received
0812 11:14:20.136708 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:20.136727 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:20.137042 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:20.137425 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:20.138206 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:20.138210 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:20.138221 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:14:20.138230 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:20.138289 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:14:20.138307 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:20.138391 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:20.138540 (+ 149us) server_negotiation.cc:299] Negotiation successful
0812 11:14:20.138718 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":129,"threads_started":1}
I20260812 11:14:20.142868 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:14:20.143579 18918 raft_consensus.cc:348] T 00000000000000000000000000000000 P f302a3d23eb84c8d92ca371d7ff27657 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f302a3d23eb84c8d92ca371d7ff27657" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45795 } }
I20260812 11:14:20.143875 18918 raft_consensus.cc:374] T 00000000000000000000000000000000 P f302a3d23eb84c8d92ca371d7ff27657 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:14:20.143929 18918 raft_consensus.cc:729] T 00000000000000000000000000000000 P f302a3d23eb84c8d92ca371d7ff27657 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f302a3d23eb84c8d92ca371d7ff27657, State: Initialized, Role: FOLLOWER
I20260812 11:14:20.144435 18918 consensus_queue.cc:260] T 00000000000000000000000000000000 P f302a3d23eb84c8d92ca371d7ff27657 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f302a3d23eb84c8d92ca371d7ff27657" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45795 } }
I20260812 11:14:20.144611 18918 raft_consensus.cc:388] T 00000000000000000000000000000000 P f302a3d23eb84c8d92ca371d7ff27657 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:14:20.144686 18918 raft_consensus.cc:482] T 00000000000000000000000000000000 P f302a3d23eb84c8d92ca371d7ff27657 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:14:20.144788 18918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f302a3d23eb84c8d92ca371d7ff27657 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:14:20.145812 18918 raft_consensus.cc:504] T 00000000000000000000000000000000 P f302a3d23eb84c8d92ca371d7ff27657 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f302a3d23eb84c8d92ca371d7ff27657" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45795 } }
I20260812 11:14:20.146145 18918 leader_election.cc:302] T 00000000000000000000000000000000 P f302a3d23eb84c8d92ca371d7ff27657 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f302a3d23eb84c8d92ca371d7ff27657; no voters:
I20260812 11:14:20.146451 18918 leader_election.cc:288] T 00000000000000000000000000000000 P f302a3d23eb84c8d92ca371d7ff27657 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:14:20.146567 18932 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f302a3d23eb84c8d92ca371d7ff27657 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:14:20.146979 18932 raft_consensus.cc:686] T 00000000000000000000000000000000 P f302a3d23eb84c8d92ca371d7ff27657 [term 1 LEADER]: Becoming Leader. State: Replica: f302a3d23eb84c8d92ca371d7ff27657, State: Running, Role: LEADER
I20260812 11:14:20.147557 18921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:20.145233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.234:41193 (local address 127.0.0.1:45795)
0812 11:14:20.145366 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:20.145371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:20.145585 (+ 214us) server_negotiation.cc:407] Connection header received
0812 11:14:20.145703 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:20.145706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:20.145736 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:20.145837 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:20.146595 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:20.146598 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:20.146601 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:14:20.146614 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:20.146644 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:14:20.146659 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:20.146722 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:20.147133 (+ 411us) server_negotiation.cc:299] Negotiation successful
0812 11:14:20.147178 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 11:14:20.147639 18932 consensus_queue.cc:237] T 00000000000000000000000000000000 P f302a3d23eb84c8d92ca371d7ff27657 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f302a3d23eb84c8d92ca371d7ff27657" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45795 } }
I20260812 11:14:20.147903 18918 sys_catalog.cc:564] T 00000000000000000000000000000000 P f302a3d23eb84c8d92ca371d7ff27657 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:14:20.149744 18934 sys_catalog.cc:455] T 00000000000000000000000000000000 P f302a3d23eb84c8d92ca371d7ff27657 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f302a3d23eb84c8d92ca371d7ff27657" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f302a3d23eb84c8d92ca371d7ff27657" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45795 } } }
I20260812 11:14:20.149720 18933 sys_catalog.cc:455] T 00000000000000000000000000000000 P f302a3d23eb84c8d92ca371d7ff27657 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f302a3d23eb84c8d92ca371d7ff27657. Latest consensus state: current_term: 1 leader_uuid: "f302a3d23eb84c8d92ca371d7ff27657" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f302a3d23eb84c8d92ca371d7ff27657" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45795 } } }
I20260812 11:14:20.149858 18934 sys_catalog.cc:458] T 00000000000000000000000000000000 P f302a3d23eb84c8d92ca371d7ff27657 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:20.149858 18933 sys_catalog.cc:458] T 00000000000000000000000000000000 P f302a3d23eb84c8d92ca371d7ff27657 [sys.catalog]: This master's current role is: LEADER
W20260812 11:14:20.152515 18958 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f302a3d23eb84c8d92ca371d7ff27657: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:14:20.152611 18958 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:14:20.152678 18949 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:14:20.153376 18949 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:14:20.158694 18949 catalog_manager.cc:1347] Generated new cluster ID: e76cf3ba8862438485a6d0d90886208e
I20260812 11:14:20.158761 18949 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:14:20.173383 18949 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:14:20.173717 18949 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:14:20.183480 18949 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f302a3d23eb84c8d92ca371d7ff27657: Generated new TSK 0
I20260812 11:14:20.184171 18949 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:14:20.195860 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 18805
2026-08-12T11:14:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1691 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:14:20 dist-test-slave-drl0 krb5kdc[18967](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:14:20 dist-test-slave-drl0 krb5kdc[18967](info): set up 2 sockets
Aug 12 11:14:20 dist-test-slave-drl0 krb5kdc[18967](info): commencing operation
krb5kdc: starting...
W20260812 11:14:21.485848 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:14:21 dist-test-slave-drl0 krb5kdc[18967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:14:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:14:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:14:21.627077 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:34033
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:43869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:34033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:14:21.754035 18991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:14:21.754319 18991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:14:21.754364 18991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:14:21.758460 18991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:14:21.758522 18991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:14:21.758543 18991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:14:21.758558 18991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:14:21.758574 18991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:14:21.763808 18991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:43869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:34033
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:34033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:14:21.765149 18991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:14:21.766166 18991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:14:21.775094 19002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:21.775192 19004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:21.775322 18991 server_base.cc:1034] running on GCE node
W20260812 11:14:21.775182 18999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:21.775988 18991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:14:21.777011 18991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:14:21.778194 18991 hybrid_clock.cc:648] HybridClock initialized: now 1786533261778168 us; error 40 us; skew 500 ppm
Aug 12 11:14:21 dist-test-slave-drl0 krb5kdc[18967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533261, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:14:21.788395 18991 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:14:21.789644 18991 webserver.cc:460] Webserver started at http://127.17.44.254:43883/ using document root <none> and password file <none>
I20260812 11:14:21.790261 18991 fs_manager.cc:359] Metadata directory not provided
I20260812 11:14:21.790370 18991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:14:21.790787 18991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:14:21.793699 18991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "c893670b32764dc6a9a2db9e6eec42d2"
format_stamp: "Formatted at 2026-08-12 11:14:21 on dist-test-slave-drl0"
server_key: "bbe5c9af2b5d771bc914047e12828310"
server_key_iv: "dd94ef5146a658998b9cd9a5e2449307"
server_key_version: "encryptionkey@0"
I20260812 11:14:21.794625 18991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "c893670b32764dc6a9a2db9e6eec42d2"
format_stamp: "Formatted at 2026-08-12 11:14:21 on dist-test-slave-drl0"
server_key: "bbe5c9af2b5d771bc914047e12828310"
server_key_iv: "dd94ef5146a658998b9cd9a5e2449307"
server_key_version: "encryptionkey@0"
I20260812 11:14:21.798831 18991 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 11:14:21.801748 19019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:14:21.803292 18991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 11:14:21.803459 18991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "c893670b32764dc6a9a2db9e6eec42d2"
format_stamp: "Formatted at 2026-08-12 11:14:21 on dist-test-slave-drl0"
server_key: "bbe5c9af2b5d771bc914047e12828310"
server_key_iv: "dd94ef5146a658998b9cd9a5e2449307"
server_key_version: "encryptionkey@0"
I20260812 11:14:21.803583 18991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:14:21.838233 18991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:14:21.841847 18991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:14:21.842063 18991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:14:21.850643 18991 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:34033
I20260812 11:14:21.850668 19112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:34033 every 8 connection(s)
I20260812 11:14:21.851630 18991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:14:21.854638 19113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:14:21.855773 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 18991
I20260812 11:14:21.855947 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:14:21.856284 17587 external_mini_cluster.cc:1428] Setting key 91cfe38501775d31e33e2e5438a8a93a
I20260812 11:14:21.861624 19113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c893670b32764dc6a9a2db9e6eec42d2: Bootstrap starting.
I20260812 11:14:21.864898 19113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c893670b32764dc6a9a2db9e6eec42d2: Neither blocks nor log segments found. Creating new log.
I20260812 11:14:21.865828 19113 log.cc:824] T 00000000000000000000000000000000 P c893670b32764dc6a9a2db9e6eec42d2: Log is configured to *not* fsync() on all Append() calls
Aug 12 11:14:21 dist-test-slave-drl0 krb5kdc[18967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:14:21.868398 19113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c893670b32764dc6a9a2db9e6eec42d2: No bootstrap required, opened a new log
I20260812 11:14:21.872941 19113 raft_consensus.cc:348] T 00000000000000000000000000000000 P c893670b32764dc6a9a2db9e6eec42d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c893670b32764dc6a9a2db9e6eec42d2" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 34033 } }
I20260812 11:14:21.873303 19113 raft_consensus.cc:374] T 00000000000000000000000000000000 P c893670b32764dc6a9a2db9e6eec42d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:14:21.873384 19113 raft_consensus.cc:729] T 00000000000000000000000000000000 P c893670b32764dc6a9a2db9e6eec42d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c893670b32764dc6a9a2db9e6eec42d2, State: Initialized, Role: FOLLOWER
I20260812 11:14:21.873898 19113 consensus_queue.cc:260] T 00000000000000000000000000000000 P c893670b32764dc6a9a2db9e6eec42d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c893670b32764dc6a9a2db9e6eec42d2" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 34033 } }
I20260812 11:14:21.874066 19113 raft_consensus.cc:388] T 00000000000000000000000000000000 P c893670b32764dc6a9a2db9e6eec42d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:14:21.874166 19113 raft_consensus.cc:482] T 00000000000000000000000000000000 P c893670b32764dc6a9a2db9e6eec42d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:14:21.874297 19113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c893670b32764dc6a9a2db9e6eec42d2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:14:21.874356 19117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:21.858061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44591 (local address 127.17.44.254:34033)
0812 11:14:21.858544 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:21.858559 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:21.858595 (+ 36us) server_negotiation.cc:407] Connection header received
0812 11:14:21.859306 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:21.859324 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:21.859663 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:21.859995 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:21.861294 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:21.862364 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:21.863503 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:21.864192 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:21.864404 (+ 212us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:21.868008 (+ 3604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:21.868013 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:21.868027 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:21.868064 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:21.870086 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:21.870874 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:21.870883 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:21.870890 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:21.871171 (+ 281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:21.871790 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:21.871795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:21.871797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:21.872469 (+ 672us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:21.872721 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:21.873128 (+ 407us) server_negotiation.cc:299] Negotiation successful
0812 11:14:21.873438 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":143,"threads_started":1}
I20260812 11:14:21.875603 19113 raft_consensus.cc:504] T 00000000000000000000000000000000 P c893670b32764dc6a9a2db9e6eec42d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c893670b32764dc6a9a2db9e6eec42d2" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 34033 } }
I20260812 11:14:21.875973 19113 leader_election.cc:302] T 00000000000000000000000000000000 P c893670b32764dc6a9a2db9e6eec42d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c893670b32764dc6a9a2db9e6eec42d2; no voters:
I20260812 11:14:21.876369 19113 leader_election.cc:288] T 00000000000000000000000000000000 P c893670b32764dc6a9a2db9e6eec42d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:14:21.876528 19119 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c893670b32764dc6a9a2db9e6eec42d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:14:21.876812 19119 raft_consensus.cc:686] T 00000000000000000000000000000000 P c893670b32764dc6a9a2db9e6eec42d2 [term 1 LEADER]: Becoming Leader. State: Replica: c893670b32764dc6a9a2db9e6eec42d2, State: Running, Role: LEADER
I20260812 11:14:21.877213 19119 consensus_queue.cc:237] T 00000000000000000000000000000000 P c893670b32764dc6a9a2db9e6eec42d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c893670b32764dc6a9a2db9e6eec42d2" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 34033 } }
I20260812 11:14:21.877738 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:14:21.877835 19113 sys_catalog.cc:564] T 00000000000000000000000000000000 P c893670b32764dc6a9a2db9e6eec42d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:14:21.879139 19120 sys_catalog.cc:455] T 00000000000000000000000000000000 P c893670b32764dc6a9a2db9e6eec42d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c893670b32764dc6a9a2db9e6eec42d2. Latest consensus state: current_term: 1 leader_uuid: "c893670b32764dc6a9a2db9e6eec42d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c893670b32764dc6a9a2db9e6eec42d2" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 34033 } } }
I20260812 11:14:21.879136 19121 sys_catalog.cc:455] T 00000000000000000000000000000000 P c893670b32764dc6a9a2db9e6eec42d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c893670b32764dc6a9a2db9e6eec42d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c893670b32764dc6a9a2db9e6eec42d2" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 34033 } } }
I20260812 11:14:21.879311 19120 sys_catalog.cc:458] T 00000000000000000000000000000000 P c893670b32764dc6a9a2db9e6eec42d2 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:21.879311 19121 sys_catalog.cc:458] T 00000000000000000000000000000000 P c893670b32764dc6a9a2db9e6eec42d2 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:21.879889 19136 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:14:21.883935 19136 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:14:21.890273 19136 catalog_manager.cc:1347] Generated new cluster ID: c6c38fe201a74845b3947d8b693f7f03
I20260812 11:14:21.890345 19136 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:14:21.891460 19117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:21.880393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33485 (local address 127.17.44.254:34033)
0812 11:14:21.880516 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:21.880521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:21.880921 (+ 400us) server_negotiation.cc:407] Connection header received
0812 11:14:21.881080 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:21.881083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:21.881131 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:21.881233 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:21.882460 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:21.883234 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:21.884469 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:21.884923 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:21.885048 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:21.886975 (+ 1927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:21.886980 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:21.886983 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:21.887010 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:21.889311 (+ 2301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:21.889913 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:21.889917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:21.889919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:21.890013 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:21.890476 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:21.890479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:21.890481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:21.890716 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:21.890887 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:21.891221 (+ 334us) server_negotiation.cc:299] Negotiation successful
0812 11:14:21.891314 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":39}
I20260812 11:14:21.895380 19136 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:14:21.896567 19136 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:14:21.908934 19136 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c893670b32764dc6a9a2db9e6eec42d2: Generated new TSK 0
I20260812 11:14:21.909765 19136 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:14:21.918236 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 18991
W20260812 11:14:21.935017 18963 connection.cc:535] client connection to 127.17.44.254:34033 recv error: Network error: failed to read from TLS socket (remote: 127.17.44.254:34033): Connection reset by peer (error 104)
2026-08-12T11:14:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1724 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:14:21 dist-test-slave-drl0 krb5kdc[19154](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:14:21 dist-test-slave-drl0 krb5kdc[19154](info): set up 2 sockets
Aug 12 11:14:21 dist-test-slave-drl0 krb5kdc[19154](info): commencing operation
krb5kdc: starting...
W20260812 11:14:23.220886 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:14:23 dist-test-slave-drl0 krb5kdc[19154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:14:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:14:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:14:23.362272 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:45323
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:33363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:45323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:14:23.484834 19177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:14:23.485157 19177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:14:23.485235 19177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:14:23.490298 19177 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:14:23.490413 19177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:14:23.490466 19177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:14:23.490525 19177 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:14:23.490563 19177 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:14:23.497638 19177 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:33363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:45323
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:45323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:14:23.498791 19177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:14:23.499882 19177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:14:23.509259 19187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:23.509390 19183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:23.509362 19177 server_base.cc:1034] running on GCE node
W20260812 11:14:23.509279 19185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:23.510056 19177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:14:23.511297 19177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:14:23.512508 19177 hybrid_clock.cc:648] HybridClock initialized: now 1786533263512450 us; error 74 us; skew 500 ppm
Aug 12 11:14:23 dist-test-slave-drl0 krb5kdc[19154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533263, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:14:23.516032 19177 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:14:23.517052 19177 webserver.cc:460] Webserver started at http://127.17.44.254:46161/ using document root <none> and password file <none>
I20260812 11:14:23.517601 19177 fs_manager.cc:359] Metadata directory not provided
I20260812 11:14:23.517663 19177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:14:23.517838 19177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:14:23.520043 19177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "43b5fece7f164fd1aea049956f71b587"
format_stamp: "Formatted at 2026-08-12 11:14:23 on dist-test-slave-drl0"
server_key: "07e8a002a3b6b98ec9d835cb0a4a3a40"
server_key_iv: "a3442c468f01e8a7ff4c46bf374c877c"
server_key_version: "encryptionkey@0"
I20260812 11:14:23.520578 19177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "43b5fece7f164fd1aea049956f71b587"
format_stamp: "Formatted at 2026-08-12 11:14:23 on dist-test-slave-drl0"
server_key: "07e8a002a3b6b98ec9d835cb0a4a3a40"
server_key_iv: "a3442c468f01e8a7ff4c46bf374c877c"
server_key_version: "encryptionkey@0"
I20260812 11:14:23.524708 19177 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 11:14:23.527772 19196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:14:23.529213 19177 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:14:23.529335 19177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "43b5fece7f164fd1aea049956f71b587"
format_stamp: "Formatted at 2026-08-12 11:14:23 on dist-test-slave-drl0"
server_key: "07e8a002a3b6b98ec9d835cb0a4a3a40"
server_key_iv: "a3442c468f01e8a7ff4c46bf374c877c"
server_key_version: "encryptionkey@0"
I20260812 11:14:23.529429 19177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:14:23.547086 19177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:14:23.551234 19177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:14:23.551414 19177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:14:23.559571 19177 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:45323
I20260812 11:14:23.559707 19275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:45323 every 8 connection(s)
I20260812 11:14:23.560597 19177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:14:23.565164 19276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:14:23.570824 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 19177
I20260812 11:14:23.571017 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:14:23.571316 17587 external_mini_cluster.cc:1428] Setting key 2dc28a28899c93a4e3f21fe12060106a
I20260812 11:14:23.573355 19276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b5fece7f164fd1aea049956f71b587: Bootstrap starting.
I20260812 11:14:23.575906 19276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43b5fece7f164fd1aea049956f71b587: Neither blocks nor log segments found. Creating new log.
I20260812 11:14:23.576781 19276 log.cc:824] T 00000000000000000000000000000000 P 43b5fece7f164fd1aea049956f71b587: Log is configured to *not* fsync() on all Append() calls
I20260812 11:14:23.579720 19276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b5fece7f164fd1aea049956f71b587: No bootstrap required, opened a new log
I20260812 11:14:23.581444 19281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:23.572982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39943 (local address 127.17.44.254:45323)
0812 11:14:23.573451 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:23.573472 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:23.573525 (+ 53us) server_negotiation.cc:407] Connection header received
0812 11:14:23.574216 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:23.574232 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:23.574581 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:23.574864 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:23.575288 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:23.576456 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:23.577606 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:23.578197 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:23.578328 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:23.579590 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:23.579595 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:23.579615 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:14:23.580084 (+ 469us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:23.580140 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:14:23.580164 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:23.580274 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:23.580558 (+ 284us) server_negotiation.cc:299] Negotiation successful
0812 11:14:23.580709 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":135,"threads_started":1}
I20260812 11:14:23.584010 19276 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43b5fece7f164fd1aea049956f71b587 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b5fece7f164fd1aea049956f71b587" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 45323 } }
I20260812 11:14:23.584276 19276 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43b5fece7f164fd1aea049956f71b587 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:14:23.584326 19276 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43b5fece7f164fd1aea049956f71b587 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43b5fece7f164fd1aea049956f71b587, State: Initialized, Role: FOLLOWER
I20260812 11:14:23.584707 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:14:23.584802 19276 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43b5fece7f164fd1aea049956f71b587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b5fece7f164fd1aea049956f71b587" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 45323 } }
I20260812 11:14:23.584949 19276 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43b5fece7f164fd1aea049956f71b587 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:14:23.585016 19276 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43b5fece7f164fd1aea049956f71b587 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:14:23.585099 19276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43b5fece7f164fd1aea049956f71b587 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:14:23.586475 19276 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43b5fece7f164fd1aea049956f71b587 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b5fece7f164fd1aea049956f71b587" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 45323 } }
I20260812 11:14:23.586850 19276 leader_election.cc:302] T 00000000000000000000000000000000 P 43b5fece7f164fd1aea049956f71b587 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43b5fece7f164fd1aea049956f71b587; no voters:
I20260812 11:14:23.587273 19276 leader_election.cc:288] T 00000000000000000000000000000000 P 43b5fece7f164fd1aea049956f71b587 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:14:23.587350 19289 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43b5fece7f164fd1aea049956f71b587 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:14:23.587605 19289 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43b5fece7f164fd1aea049956f71b587 [term 1 LEADER]: Becoming Leader. State: Replica: 43b5fece7f164fd1aea049956f71b587, State: Running, Role: LEADER
I20260812 11:14:23.587958 19289 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43b5fece7f164fd1aea049956f71b587 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b5fece7f164fd1aea049956f71b587" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 45323 } }
I20260812 11:14:23.588641 19276 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43b5fece7f164fd1aea049956f71b587 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:14:23.589701 19293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b5fece7f164fd1aea049956f71b587 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43b5fece7f164fd1aea049956f71b587. Latest consensus state: current_term: 1 leader_uuid: "43b5fece7f164fd1aea049956f71b587" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b5fece7f164fd1aea049956f71b587" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 45323 } } }
I20260812 11:14:23.589824 19293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b5fece7f164fd1aea049956f71b587 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:23.590190 19292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b5fece7f164fd1aea049956f71b587 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43b5fece7f164fd1aea049956f71b587" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b5fece7f164fd1aea049956f71b587" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 45323 } } }
I20260812 11:14:23.590380 19292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b5fece7f164fd1aea049956f71b587 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:23.590975 19306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:14:23.592252 19281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:23.587328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55465 (local address 127.17.44.254:45323)
0812 11:14:23.587451 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:23.587456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:23.587673 (+ 217us) server_negotiation.cc:407] Connection header received
0812 11:14:23.587868 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:23.587871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:23.587919 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:23.587971 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:23.588347 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:23.588989 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:23.590144 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:23.590484 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:23.590575 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:23.591781 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:23.591785 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:23.591787 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:14:23.591802 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:23.591839 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:14:23.591853 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:23.591953 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:23.592091 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 11:14:23.592124 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 11:14:23.594283 19306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:14:23.601320 19306 catalog_manager.cc:1347] Generated new cluster ID: e4ae8adffefb4e51953f754ad226cf26
I20260812 11:14:23.601382 19306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:14:23.621521 19306 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:14:23.622786 19306 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:14:23.632927 19306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43b5fece7f164fd1aea049956f71b587: Generated new TSK 0
I20260812 11:14:23.633847 19306 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:14:23.643304 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 19177
W20260812 11:14:23.662182 19152 connection.cc:535] client connection to 127.17.44.254:45323 recv error: Network error: failed to read from TLS socket (remote: 127.17.44.254:45323): Connection reset by peer (error 104)
2026-08-12T11:14:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1727 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:14:23 dist-test-slave-drl0 krb5kdc[19320](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:14:23 dist-test-slave-drl0 krb5kdc[19320](info): set up 2 sockets
Aug 12 11:14:23 dist-test-slave-drl0 krb5kdc[19320](info): commencing operation
krb5kdc: starting...
W20260812 11:14:24.945312 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.000s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:14:25 dist-test-slave-drl0 krb5kdc[19320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:14:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:14:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:14:25.094266 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:36955
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:35423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:36955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:14:25.223189 19340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:14:25.223613 19340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:14:25.223699 19340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:14:25.229399 19340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:14:25.229557 19340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:14:25.229599 19340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:14:25.229641 19340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:14:25.229678 19340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:14:25.237960 19340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:35423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:36955
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:36955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:14:25.239575 19340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:14:25.241277 19340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:14:25.252641 19347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:25.252641 19346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:25.252877 19350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:25.253300 19340 server_base.cc:1034] running on GCE node
I20260812 11:14:25.253961 19340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:14:25.255262 19340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:14:25.256496 19340 hybrid_clock.cc:648] HybridClock initialized: now 1786533265256470 us; error 38 us; skew 500 ppm
Aug 12 11:14:25 dist-test-slave-drl0 krb5kdc[19320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533265, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:14:25.259639 19340 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:14:25.260557 19340 webserver.cc:460] Webserver started at http://127.17.44.254:38709/ using document root <none> and password file <none>
I20260812 11:14:25.261174 19340 fs_manager.cc:359] Metadata directory not provided
I20260812 11:14:25.261278 19340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:14:25.261520 19340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:14:25.263875 19340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "5d628801dcbf4f2dae320c07fb05217c"
format_stamp: "Formatted at 2026-08-12 11:14:25 on dist-test-slave-drl0"
server_key: "260dd4d162a85e56c16212b67736736e"
server_key_iv: "446a879aae6c154952ea87e19d68c9b9"
server_key_version: "encryptionkey@0"
I20260812 11:14:25.264583 19340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "5d628801dcbf4f2dae320c07fb05217c"
format_stamp: "Formatted at 2026-08-12 11:14:25 on dist-test-slave-drl0"
server_key: "260dd4d162a85e56c16212b67736736e"
server_key_iv: "446a879aae6c154952ea87e19d68c9b9"
server_key_version: "encryptionkey@0"
I20260812 11:14:25.268512 19340 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 11:14:25.271210 19362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:14:25.273164 19340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 11:14:25.273343 19340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "5d628801dcbf4f2dae320c07fb05217c"
format_stamp: "Formatted at 2026-08-12 11:14:25 on dist-test-slave-drl0"
server_key: "260dd4d162a85e56c16212b67736736e"
server_key_iv: "446a879aae6c154952ea87e19d68c9b9"
server_key_version: "encryptionkey@0"
I20260812 11:14:25.273469 19340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:14:25.278407 19340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:14:25.281847 19340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:14:25.282058 19340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:14:25.291172 19340 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:36955
I20260812 11:14:25.291198 19438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:36955 every 8 connection(s)
I20260812 11:14:25.292132 19340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:14:25.293632 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 19340
I20260812 11:14:25.293814 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:14:25.294112 17587 external_mini_cluster.cc:1428] Setting key 0c27fefb4882747ceb48389c5d1c5944
I20260812 11:14:25.295190 19439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:14:25.299384 19442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:25.295593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43875 (local address 127.17.44.254:36955)
0812 11:14:25.296075 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:25.296090 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:25.296124 (+ 34us) server_negotiation.cc:407] Connection header received
0812 11:14:25.296784 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:25.296799 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:25.297058 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:25.297395 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:25.298186 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:25.298190 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:25.298201 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:14:25.298223 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:25.298274 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:14:25.298294 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:25.298442 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:25.298512 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 11:14:25.298665 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":160,"threads_started":1}
I20260812 11:14:25.301831 19439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d628801dcbf4f2dae320c07fb05217c: Bootstrap starting.
I20260812 11:14:25.302798 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:14:25.304486 19439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d628801dcbf4f2dae320c07fb05217c: Neither blocks nor log segments found. Creating new log.
I20260812 11:14:25.305320 19439 log.cc:824] T 00000000000000000000000000000000 P 5d628801dcbf4f2dae320c07fb05217c: Log is configured to *not* fsync() on all Append() calls
I20260812 11:14:25.307222 19442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:25.305080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38763 (local address 127.17.44.254:36955)
0812 11:14:25.305215 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:25.305219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:25.305461 (+ 242us) server_negotiation.cc:407] Connection header received
0812 11:14:25.305523 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:25.305526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:25.305560 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:25.305639 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:25.306569 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:25.306574 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:25.306576 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:14:25.306600 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:25.306652 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:14:25.306668 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:25.306801 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:25.307029 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 11:14:25.307083 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 11:14:25.307627 19439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d628801dcbf4f2dae320c07fb05217c: No bootstrap required, opened a new log
I20260812 11:14:25.311388 19439 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d628801dcbf4f2dae320c07fb05217c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d628801dcbf4f2dae320c07fb05217c" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36955 } }
I20260812 11:14:25.311602 19439 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d628801dcbf4f2dae320c07fb05217c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:14:25.311638 19439 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d628801dcbf4f2dae320c07fb05217c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d628801dcbf4f2dae320c07fb05217c, State: Initialized, Role: FOLLOWER
I20260812 11:14:25.312136 19439 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d628801dcbf4f2dae320c07fb05217c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d628801dcbf4f2dae320c07fb05217c" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36955 } }
I20260812 11:14:25.312325 19439 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d628801dcbf4f2dae320c07fb05217c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:14:25.312455 19439 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d628801dcbf4f2dae320c07fb05217c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:14:25.312587 19439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d628801dcbf4f2dae320c07fb05217c [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:14:25.313510 19439 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d628801dcbf4f2dae320c07fb05217c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d628801dcbf4f2dae320c07fb05217c" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36955 } }
I20260812 11:14:25.313860 19439 leader_election.cc:302] T 00000000000000000000000000000000 P 5d628801dcbf4f2dae320c07fb05217c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d628801dcbf4f2dae320c07fb05217c; no voters:
I20260812 11:14:25.314183 19439 leader_election.cc:288] T 00000000000000000000000000000000 P 5d628801dcbf4f2dae320c07fb05217c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:14:25.314363 19458 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d628801dcbf4f2dae320c07fb05217c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:14:25.314638 19458 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d628801dcbf4f2dae320c07fb05217c [term 1 LEADER]: Becoming Leader. State: Replica: 5d628801dcbf4f2dae320c07fb05217c, State: Running, Role: LEADER
I20260812 11:14:25.314992 19458 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d628801dcbf4f2dae320c07fb05217c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d628801dcbf4f2dae320c07fb05217c" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36955 } }
I20260812 11:14:25.315524 19439 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d628801dcbf4f2dae320c07fb05217c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:14:25.316689 19461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d628801dcbf4f2dae320c07fb05217c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d628801dcbf4f2dae320c07fb05217c. Latest consensus state: current_term: 1 leader_uuid: "5d628801dcbf4f2dae320c07fb05217c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d628801dcbf4f2dae320c07fb05217c" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36955 } } }
I20260812 11:14:25.316833 19459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d628801dcbf4f2dae320c07fb05217c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d628801dcbf4f2dae320c07fb05217c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d628801dcbf4f2dae320c07fb05217c" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 36955 } } }
I20260812 11:14:25.316833 19461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d628801dcbf4f2dae320c07fb05217c [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:25.316927 19459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d628801dcbf4f2dae320c07fb05217c [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:25.317358 19470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:14:25.320686 19470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:14:25.327952 19470 catalog_manager.cc:1347] Generated new cluster ID: 2b875d33e9ec482aaee376ca3f359e2a
I20260812 11:14:25.328048 19470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:14:25.354348 19470 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:14:25.354701 19470 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:14:25.361066 19470 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d628801dcbf4f2dae320c07fb05217c: Generated new TSK 0
I20260812 11:14:25.362435 19470 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:14:25.375463 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 19340
2026-08-12T11:14:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1730 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:14:25 dist-test-slave-drl0 krb5kdc[19497](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:14:25 dist-test-slave-drl0 krb5kdc[19497](info): set up 2 sockets
Aug 12 11:14:25 dist-test-slave-drl0 krb5kdc[19497](info): commencing operation
krb5kdc: starting...
W20260812 11:14:26.673498 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:14:26 dist-test-slave-drl0 krb5kdc[19497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:14:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:14:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:14:26.808063 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:37171
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:37391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:37171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:14:26.934118 19513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:14:26.934584 19513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:14:26.934670 19513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:14:26.940567 19513 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:14:26.940649 19513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:14:26.940670 19513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:14:26.940689 19513 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:14:26.940707 19513 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:14:26.946902 19513 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:37391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:37171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:37171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:14:26.948148 19513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:14:26.949226 19513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:14:26.958662 19522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:26.958635 19525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:26.958662 19520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:26.959291 19513 server_base.cc:1034] running on GCE node
I20260812 11:14:26.960135 19513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:14:26.961318 19513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:14:26.962543 19513 hybrid_clock.cc:648] HybridClock initialized: now 1786533266962477 us; error 68 us; skew 500 ppm
Aug 12 11:14:26 dist-test-slave-drl0 krb5kdc[19497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533266, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:14:26.970320 19513 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:14:26.971374 19513 webserver.cc:460] Webserver started at http://127.17.44.254:43419/ using document root <none> and password file <none>
I20260812 11:14:26.971946 19513 fs_manager.cc:359] Metadata directory not provided
I20260812 11:14:26.972008 19513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:14:26.972182 19513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:14:26.974299 19513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "a94a8911abd94c939172ff6bbe2106c4"
format_stamp: "Formatted at 2026-08-12 11:14:26 on dist-test-slave-drl0"
server_key: "4798ed636a014248afb589e01fe8f188"
server_key_iv: "78305aaf9eba5d0b4a7070c8149d1230"
server_key_version: "encryptionkey@0"
I20260812 11:14:26.974872 19513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "a94a8911abd94c939172ff6bbe2106c4"
format_stamp: "Formatted at 2026-08-12 11:14:26 on dist-test-slave-drl0"
server_key: "4798ed636a014248afb589e01fe8f188"
server_key_iv: "78305aaf9eba5d0b4a7070c8149d1230"
server_key_version: "encryptionkey@0"
I20260812 11:14:26.979681 19513 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 11:14:26.982779 19535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:14:26.984683 19513 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 11:14:26.984892 19513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "a94a8911abd94c939172ff6bbe2106c4"
format_stamp: "Formatted at 2026-08-12 11:14:26 on dist-test-slave-drl0"
server_key: "4798ed636a014248afb589e01fe8f188"
server_key_iv: "78305aaf9eba5d0b4a7070c8149d1230"
server_key_version: "encryptionkey@0"
I20260812 11:14:26.985057 19513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:14:27.038600 19513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:14:27.043411 19513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:14:27.043670 19513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:14:27.052191 19513 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:37171
I20260812 11:14:27.052238 19616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:37171 every 8 connection(s)
I20260812 11:14:27.053427 19513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:14:27.056573 19617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:14:27.057210 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 19513
I20260812 11:14:27.057370 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:14:27.057696 17587 external_mini_cluster.cc:1428] Setting key 6db2c749402b6862859fa3ca35c2dba2
I20260812 11:14:27.063990 19617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a94a8911abd94c939172ff6bbe2106c4: Bootstrap starting.
I20260812 11:14:27.067164 19617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a94a8911abd94c939172ff6bbe2106c4: Neither blocks nor log segments found. Creating new log.
I20260812 11:14:27.067971 19617 log.cc:824] T 00000000000000000000000000000000 P a94a8911abd94c939172ff6bbe2106c4: Log is configured to *not* fsync() on all Append() calls
Aug 12 11:14:27 dist-test-slave-drl0 krb5kdc[19497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:14:27.071204 19617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a94a8911abd94c939172ff6bbe2106c4: No bootstrap required, opened a new log
I20260812 11:14:27.075538 19624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:27.059627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50397 (local address 127.17.44.254:37171)
0812 11:14:27.060197 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:27.060212 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:27.060255 (+ 43us) server_negotiation.cc:407] Connection header received
0812 11:14:27.060897 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:27.060913 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:27.061271 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:27.061655 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:27.062885 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:27.064169 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:27.065120 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:27.065655 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:27.065847 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:27.069342 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:27.069349 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:27.069367 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:27.069399 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:27.071917 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:27.072604 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:27.072609 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:27.072615 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:27.072736 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:27.073083 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:27.073085 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:27.073087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:27.073655 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:27.073821 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:27.074259 (+ 438us) server_negotiation.cc:299] Negotiation successful
0812 11:14:27.074611 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":162,"threads_started":1}
I20260812 11:14:27.076313 19617 raft_consensus.cc:348] T 00000000000000000000000000000000 P a94a8911abd94c939172ff6bbe2106c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94a8911abd94c939172ff6bbe2106c4" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 37171 } }
I20260812 11:14:27.076591 19617 raft_consensus.cc:374] T 00000000000000000000000000000000 P a94a8911abd94c939172ff6bbe2106c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:14:27.076670 19617 raft_consensus.cc:729] T 00000000000000000000000000000000 P a94a8911abd94c939172ff6bbe2106c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a94a8911abd94c939172ff6bbe2106c4, State: Initialized, Role: FOLLOWER
I20260812 11:14:27.077143 19617 consensus_queue.cc:260] T 00000000000000000000000000000000 P a94a8911abd94c939172ff6bbe2106c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94a8911abd94c939172ff6bbe2106c4" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 37171 } }
I20260812 11:14:27.077338 19617 raft_consensus.cc:388] T 00000000000000000000000000000000 P a94a8911abd94c939172ff6bbe2106c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:14:27.077440 19617 raft_consensus.cc:482] T 00000000000000000000000000000000 P a94a8911abd94c939172ff6bbe2106c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:14:27.077572 19617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a94a8911abd94c939172ff6bbe2106c4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:14:27.078642 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:14:27.078902 19617 raft_consensus.cc:504] T 00000000000000000000000000000000 P a94a8911abd94c939172ff6bbe2106c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94a8911abd94c939172ff6bbe2106c4" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 37171 } }
I20260812 11:14:27.079311 19617 leader_election.cc:302] T 00000000000000000000000000000000 P a94a8911abd94c939172ff6bbe2106c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a94a8911abd94c939172ff6bbe2106c4; no voters:
I20260812 11:14:27.079658 19617 leader_election.cc:288] T 00000000000000000000000000000000 P a94a8911abd94c939172ff6bbe2106c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:14:27.079957 19626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a94a8911abd94c939172ff6bbe2106c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:14:27.080320 19626 raft_consensus.cc:686] T 00000000000000000000000000000000 P a94a8911abd94c939172ff6bbe2106c4 [term 1 LEADER]: Becoming Leader. State: Replica: a94a8911abd94c939172ff6bbe2106c4, State: Running, Role: LEADER
I20260812 11:14:27.080878 19626 consensus_queue.cc:237] T 00000000000000000000000000000000 P a94a8911abd94c939172ff6bbe2106c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94a8911abd94c939172ff6bbe2106c4" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 37171 } }
I20260812 11:14:27.081322 19617 sys_catalog.cc:564] T 00000000000000000000000000000000 P a94a8911abd94c939172ff6bbe2106c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:14:27.082739 19630 sys_catalog.cc:455] T 00000000000000000000000000000000 P a94a8911abd94c939172ff6bbe2106c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a94a8911abd94c939172ff6bbe2106c4. Latest consensus state: current_term: 1 leader_uuid: "a94a8911abd94c939172ff6bbe2106c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94a8911abd94c939172ff6bbe2106c4" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 37171 } } }
I20260812 11:14:27.082964 19630 sys_catalog.cc:458] T 00000000000000000000000000000000 P a94a8911abd94c939172ff6bbe2106c4 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:27.083254 19627 sys_catalog.cc:455] T 00000000000000000000000000000000 P a94a8911abd94c939172ff6bbe2106c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a94a8911abd94c939172ff6bbe2106c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94a8911abd94c939172ff6bbe2106c4" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 37171 } } }
I20260812 11:14:27.083354 19627 sys_catalog.cc:458] T 00000000000000000000000000000000 P a94a8911abd94c939172ff6bbe2106c4 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:27.083964 19647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:14:27.088515 19647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:14:27.092643 19624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:27.081972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39033 (local address 127.17.44.254:37171)
0812 11:14:27.082104 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:27.082112 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:27.082700 (+ 588us) server_negotiation.cc:407] Connection header received
0812 11:14:27.082972 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:27.082976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:27.083035 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:27.083101 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:27.084287 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:27.084872 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:27.086002 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:27.086246 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:27.086369 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:27.088225 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:27.088230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:27.088235 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:27.088266 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:27.089886 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:27.090570 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:27.090576 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:27.090578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:27.090673 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:27.091396 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:27.091400 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:27.091402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:27.091727 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:27.091857 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:27.092278 (+ 421us) server_negotiation.cc:299] Negotiation successful
0812 11:14:27.092462 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 11:14:27.096669 19647 catalog_manager.cc:1347] Generated new cluster ID: f1779dff581843cdbc977803401e771e
I20260812 11:14:27.096791 19647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:14:27.122426 19647 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:14:27.123972 19647 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:14:27.140215 19647 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a94a8911abd94c939172ff6bbe2106c4: Generated new TSK 0
I20260812 11:14:27.141129 19647 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:14:27.152451 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 19513
2026-08-12T11:14:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1772 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:14:27 dist-test-slave-drl0 krb5kdc[19671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:14:27 dist-test-slave-drl0 krb5kdc[19671](info): set up 2 sockets
Aug 12 11:14:27 dist-test-slave-drl0 krb5kdc[19671](info): commencing operation
krb5kdc: starting...
W20260812 11:14:28.461570 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:14:28 dist-test-slave-drl0 krb5kdc[19671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:14:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:14:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:14:28.604571 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:45459
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:37359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:45459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:14:28.731067 19692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:14:28.731496 19692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:14:28.731631 19692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:14:28.735620 19692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:14:28.735673 19692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:14:28.735693 19692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:14:28.735710 19692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:14:28.735729 19692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:14:28.741103 19692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:37359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:45459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:45459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:14:28.742199 19692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:14:28.743271 19692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:14:28.753077 19704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:28.753159 19692 server_base.cc:1034] running on GCE node
W20260812 11:14:28.753062 19699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:28.753062 19701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:28.753886 19692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:14:28.755191 19692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:14:28.756475 19692 hybrid_clock.cc:648] HybridClock initialized: now 1786533268756457 us; error 39 us; skew 500 ppm
Aug 12 11:14:28 dist-test-slave-drl0 krb5kdc[19671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533268, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:14:28.759754 19692 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:14:28.760686 19692 webserver.cc:460] Webserver started at http://127.17.44.254:41091/ using document root <none> and password file <none>
I20260812 11:14:28.761312 19692 fs_manager.cc:359] Metadata directory not provided
I20260812 11:14:28.761420 19692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:14:28.761643 19692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:14:28.763758 19692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "a5159ef581de4db284868763291b9b49"
format_stamp: "Formatted at 2026-08-12 11:14:28 on dist-test-slave-drl0"
server_key: "bea25193e6ab079900db3ca16f6a3450"
server_key_iv: "5c4ecc4254739681cc2e26ed708450be"
server_key_version: "encryptionkey@0"
I20260812 11:14:28.764328 19692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "a5159ef581de4db284868763291b9b49"
format_stamp: "Formatted at 2026-08-12 11:14:28 on dist-test-slave-drl0"
server_key: "bea25193e6ab079900db3ca16f6a3450"
server_key_iv: "5c4ecc4254739681cc2e26ed708450be"
server_key_version: "encryptionkey@0"
I20260812 11:14:28.768452 19692 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260812 11:14:28.771651 19716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:14:28.773177 19692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 11:14:28.773329 19692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "a5159ef581de4db284868763291b9b49"
format_stamp: "Formatted at 2026-08-12 11:14:28 on dist-test-slave-drl0"
server_key: "bea25193e6ab079900db3ca16f6a3450"
server_key_iv: "5c4ecc4254739681cc2e26ed708450be"
server_key_version: "encryptionkey@0"
I20260812 11:14:28.773450 19692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:14:28.802438 19692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:14:28.806049 19692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:14:28.806281 19692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:14:28.814365 19692 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:45459
I20260812 11:14:28.814374 19797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:45459 every 8 connection(s)
I20260812 11:14:28.815742 19692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:14:28.818596 19798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:14:28.822819 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 19692
I20260812 11:14:28.823017 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:14:28.823320 17587 external_mini_cluster.cc:1428] Setting key 94887bb9cc812db32af1168b45401e7a
I20260812 11:14:28.824462 19798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5159ef581de4db284868763291b9b49: Bootstrap starting.
I20260812 11:14:28.826658 19798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5159ef581de4db284868763291b9b49: Neither blocks nor log segments found. Creating new log.
I20260812 11:14:28.827770 19798 log.cc:824] T 00000000000000000000000000000000 P a5159ef581de4db284868763291b9b49: Log is configured to *not* fsync() on all Append() calls
I20260812 11:14:28.830037 19798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5159ef581de4db284868763291b9b49: No bootstrap required, opened a new log
I20260812 11:14:28.834163 19798 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5159ef581de4db284868763291b9b49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5159ef581de4db284868763291b9b49" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 45459 } }
I20260812 11:14:28.834362 19802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:28.824824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58851 (local address 127.17.44.254:45459)
0812 11:14:28.825395 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:28.825412 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:28.825450 (+ 38us) server_negotiation.cc:407] Connection header received
0812 11:14:28.826223 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:28.826244 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:28.826806 (+ 562us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:28.827177 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:28.827686 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:28.829003 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:28.830261 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:28.830722 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:28.830870 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:28.832028 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:28.832036 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:28.832058 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:14:28.832652 (+ 594us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:28.832745 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:14:28.832779 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:28.832896 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:28.833177 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 11:14:28.833612 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":168,"threads_started":1}
I20260812 11:14:28.834390 19798 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5159ef581de4db284868763291b9b49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:14:28.834574 19798 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5159ef581de4db284868763291b9b49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5159ef581de4db284868763291b9b49, State: Initialized, Role: FOLLOWER
I20260812 11:14:28.835129 19798 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5159ef581de4db284868763291b9b49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5159ef581de4db284868763291b9b49" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 45459 } }
I20260812 11:14:28.835253 19798 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5159ef581de4db284868763291b9b49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:14:28.835297 19798 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5159ef581de4db284868763291b9b49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:14:28.835362 19798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5159ef581de4db284868763291b9b49 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:14:28.836706 19798 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5159ef581de4db284868763291b9b49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5159ef581de4db284868763291b9b49" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 45459 } }
I20260812 11:14:28.837308 19798 leader_election.cc:302] T 00000000000000000000000000000000 P a5159ef581de4db284868763291b9b49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5159ef581de4db284868763291b9b49; no voters:
I20260812 11:14:28.837798 19798 leader_election.cc:288] T 00000000000000000000000000000000 P a5159ef581de4db284868763291b9b49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:14:28.837941 19804 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5159ef581de4db284868763291b9b49 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:14:28.838184 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:14:28.838220 19804 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5159ef581de4db284868763291b9b49 [term 1 LEADER]: Becoming Leader. State: Replica: a5159ef581de4db284868763291b9b49, State: Running, Role: LEADER
I20260812 11:14:28.838523 19804 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5159ef581de4db284868763291b9b49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5159ef581de4db284868763291b9b49" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 45459 } }
I20260812 11:14:28.839548 19798 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5159ef581de4db284868763291b9b49 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:14:28.839708 19810 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5159ef581de4db284868763291b9b49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5159ef581de4db284868763291b9b49. Latest consensus state: current_term: 1 leader_uuid: "a5159ef581de4db284868763291b9b49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5159ef581de4db284868763291b9b49" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 45459 } } }
I20260812 11:14:28.839843 19810 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5159ef581de4db284868763291b9b49 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:28.840118 19808 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5159ef581de4db284868763291b9b49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5159ef581de4db284868763291b9b49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5159ef581de4db284868763291b9b49" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 45459 } } }
I20260812 11:14:28.840210 19808 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5159ef581de4db284868763291b9b49 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:28.840515 19819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:14:28.843760 19819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:14:28.848011 19802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:28.842325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43081 (local address 127.17.44.254:45459)
0812 11:14:28.842433 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:28.842437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:28.842901 (+ 464us) server_negotiation.cc:407] Connection header received
0812 11:14:28.843069 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:28.843072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:28.843125 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:28.843215 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:28.843520 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:28.844494 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:28.845766 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:28.846137 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:28.846246 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:28.847414 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:28.847421 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:28.847425 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:14:28.847439 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:28.847470 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:14:28.847486 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:28.847572 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:28.847718 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 11:14:28.847790 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 11:14:28.850088 19819 catalog_manager.cc:1347] Generated new cluster ID: 1d13cf9a0d12438489e2aadd98a4e21e
I20260812 11:14:28.850162 19819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:14:28.866942 19819 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:14:28.867987 19819 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:14:28.874310 19819 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5159ef581de4db284868763291b9b49: Generated new TSK 0
I20260812 11:14:28.875067 19819 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:14:28.886103 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 19692
2026-08-12T11:14:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1738 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:14:28 dist-test-slave-drl0 krb5kdc[19844](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:14:28 dist-test-slave-drl0 krb5kdc[19844](info): set up 2 sockets
Aug 12 11:14:28 dist-test-slave-drl0 krb5kdc[19844](info): commencing operation
krb5kdc: starting...
W20260812 11:14:30.193792 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:14:30 dist-test-slave-drl0 krb5kdc[19844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:14:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:14:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:14:30.334748 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:41659
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:33417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:41659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:14:30.455730 19862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:14:30.455997 19862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:14:30.456039 19862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:14:30.460191 19862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:14:30.460258 19862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:14:30.460279 19862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:14:30.460295 19862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:14:30.460319 19862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:14:30.465775 19862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:33417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:41659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:41659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:14:30.466822 19862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:14:30.467844 19862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 11:14:30.477969 19862 server_base.cc:1034] running on GCE node
W20260812 11:14:30.477905 19871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:30.477944 19874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:30.477905 19872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:30.479282 19862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:14:30.480409 19862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:14:30.481604 19862 hybrid_clock.cc:648] HybridClock initialized: now 1786533270481564 us; error 55 us; skew 500 ppm
Aug 12 11:14:30 dist-test-slave-drl0 krb5kdc[19844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533270, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:14:30.485181 19862 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:14:30.486123 19862 webserver.cc:460] Webserver started at http://127.17.44.254:37485/ using document root <none> and password file <none>
I20260812 11:14:30.486824 19862 fs_manager.cc:359] Metadata directory not provided
I20260812 11:14:30.487000 19862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:14:30.487257 19862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:14:30.489398 19862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "416bc2d733544d9dbe7203c794c4fd76"
format_stamp: "Formatted at 2026-08-12 11:14:30 on dist-test-slave-drl0"
server_key: "0db25d1956da3b2475914758d42e701d"
server_key_iv: "a96dd6bccde2b3f3de866d92b0c120ba"
server_key_version: "encryptionkey@0"
I20260812 11:14:30.490007 19862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "416bc2d733544d9dbe7203c794c4fd76"
format_stamp: "Formatted at 2026-08-12 11:14:30 on dist-test-slave-drl0"
server_key: "0db25d1956da3b2475914758d42e701d"
server_key_iv: "a96dd6bccde2b3f3de866d92b0c120ba"
server_key_version: "encryptionkey@0"
I20260812 11:14:30.494151 19862 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 11:14:30.496878 19891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:14:30.498754 19862 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 11:14:30.499023 19862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "416bc2d733544d9dbe7203c794c4fd76"
format_stamp: "Formatted at 2026-08-12 11:14:30 on dist-test-slave-drl0"
server_key: "0db25d1956da3b2475914758d42e701d"
server_key_iv: "a96dd6bccde2b3f3de866d92b0c120ba"
server_key_version: "encryptionkey@0"
I20260812 11:14:30.499199 19862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:14:30.504757 19862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:14:30.508701 19862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:14:30.508962 19862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:14:30.518507 19862 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:41659
I20260812 11:14:30.518618 19978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:41659 every 8 connection(s)
I20260812 11:14:30.519832 19862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:14:30.522775 19979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:14:30.523343 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 19862
I20260812 11:14:30.523491 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:14:30.523795 17587 external_mini_cluster.cc:1428] Setting key 279877337cf0110e5fbb6d72fe045a37
I20260812 11:14:30.529739 19982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:30.525366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50315 (local address 127.17.44.254:41659)
0812 11:14:30.525827 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:30.525847 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:30.525889 (+ 42us) server_negotiation.cc:407] Connection header received
0812 11:14:30.526766 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:30.526791 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:30.527179 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:30.527556 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:30.528292 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:30.528301 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:30.528317 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:14:30.528329 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:30.528394 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:14:30.528422 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:30.528564 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:30.528716 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 11:14:30.528868 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":155,"threads_started":1}
I20260812 11:14:30.530752 19979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 416bc2d733544d9dbe7203c794c4fd76: Bootstrap starting.
I20260812 11:14:30.532740 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:14:30.533197 19979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 416bc2d733544d9dbe7203c794c4fd76: Neither blocks nor log segments found. Creating new log.
I20260812 11:14:30.533911 19979 log.cc:824] T 00000000000000000000000000000000 P 416bc2d733544d9dbe7203c794c4fd76: Log is configured to *not* fsync() on all Append() calls
I20260812 11:14:30.536124 19979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 416bc2d733544d9dbe7203c794c4fd76: No bootstrap required, opened a new log
I20260812 11:14:30.537559 19982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:30.535015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60419 (local address 127.17.44.254:41659)
0812 11:14:30.535136 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:30.535140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:30.535420 (+ 280us) server_negotiation.cc:407] Connection header received
0812 11:14:30.535511 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:30.535514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:30.535546 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:30.535636 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:30.537098 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:30.537101 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:30.537104 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 11:14:30.537117 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:30.537146 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 11:14:30.537161 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:30.537257 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:30.537396 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 11:14:30.537441 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 11:14:30.540284 19979 raft_consensus.cc:348] T 00000000000000000000000000000000 P 416bc2d733544d9dbe7203c794c4fd76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "416bc2d733544d9dbe7203c794c4fd76" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 41659 } }
I20260812 11:14:30.540517 19979 raft_consensus.cc:374] T 00000000000000000000000000000000 P 416bc2d733544d9dbe7203c794c4fd76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 11:14:30.540551 19979 raft_consensus.cc:729] T 00000000000000000000000000000000 P 416bc2d733544d9dbe7203c794c4fd76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 416bc2d733544d9dbe7203c794c4fd76, State: Initialized, Role: FOLLOWER
I20260812 11:14:30.541023 19979 consensus_queue.cc:260] T 00000000000000000000000000000000 P 416bc2d733544d9dbe7203c794c4fd76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "416bc2d733544d9dbe7203c794c4fd76" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 41659 } }
I20260812 11:14:30.541144 19979 raft_consensus.cc:388] T 00000000000000000000000000000000 P 416bc2d733544d9dbe7203c794c4fd76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 11:14:30.541224 19979 raft_consensus.cc:482] T 00000000000000000000000000000000 P 416bc2d733544d9dbe7203c794c4fd76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 11:14:30.541323 19979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 416bc2d733544d9dbe7203c794c4fd76 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:14:30.542243 19979 raft_consensus.cc:504] T 00000000000000000000000000000000 P 416bc2d733544d9dbe7203c794c4fd76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "416bc2d733544d9dbe7203c794c4fd76" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 41659 } }
I20260812 11:14:30.542598 19979 leader_election.cc:302] T 00000000000000000000000000000000 P 416bc2d733544d9dbe7203c794c4fd76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 416bc2d733544d9dbe7203c794c4fd76; no voters:
I20260812 11:14:30.542933 19979 leader_election.cc:288] T 00000000000000000000000000000000 P 416bc2d733544d9dbe7203c794c4fd76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 11:14:30.543045 19997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 416bc2d733544d9dbe7203c794c4fd76 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:14:30.543224 19997 raft_consensus.cc:686] T 00000000000000000000000000000000 P 416bc2d733544d9dbe7203c794c4fd76 [term 1 LEADER]: Becoming Leader. State: Replica: 416bc2d733544d9dbe7203c794c4fd76, State: Running, Role: LEADER
I20260812 11:14:30.543552 19997 consensus_queue.cc:237] T 00000000000000000000000000000000 P 416bc2d733544d9dbe7203c794c4fd76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "416bc2d733544d9dbe7203c794c4fd76" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 41659 } }
I20260812 11:14:30.544261 19979 sys_catalog.cc:564] T 00000000000000000000000000000000 P 416bc2d733544d9dbe7203c794c4fd76 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:14:30.545497 19998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 416bc2d733544d9dbe7203c794c4fd76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "416bc2d733544d9dbe7203c794c4fd76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "416bc2d733544d9dbe7203c794c4fd76" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 41659 } } }
I20260812 11:14:30.545653 19998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 416bc2d733544d9dbe7203c794c4fd76 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:30.545881 19999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 416bc2d733544d9dbe7203c794c4fd76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 416bc2d733544d9dbe7203c794c4fd76. Latest consensus state: current_term: 1 leader_uuid: "416bc2d733544d9dbe7203c794c4fd76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "416bc2d733544d9dbe7203c794c4fd76" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 41659 } } }
I20260812 11:14:30.546080 19999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 416bc2d733544d9dbe7203c794c4fd76 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:30.546648 20011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:14:30.549449 20011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:14:30.555617 20011 catalog_manager.cc:1347] Generated new cluster ID: f32eedc7c8fc479cb341723c69d8e151
I20260812 11:14:30.555787 20011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:14:30.589534 20011 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:14:30.589900 20011 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:14:30.606484 20011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 416bc2d733544d9dbe7203c794c4fd76: Generated new TSK 0
I20260812 11:14:30.607277 20011 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 11:14:30.611356 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 19862
2026-08-12T11:14:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1728 ms)
[----------] 18 tests from AuthTokenIssuingTest (31117 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:14:30 dist-test-slave-drl0 krb5kdc[20022](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:14:30 dist-test-slave-drl0 krb5kdc[20022](info): set up 2 sockets
Aug 12 11:14:30 dist-test-slave-drl0 krb5kdc[20022](info): commencing operation
krb5kdc: starting...
W20260812 11:14:31.909968 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:14:32 dist-test-slave-drl0 krb5kdc[20022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:14:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:14:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:14:32.050863 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:44427
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:46409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:44427,127.17.44.253:45173,127.17.44.252:45509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:14:32.169406 20044 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 11:14:32.169867 20044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:14:32.169925 20044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:14:32.169982 20044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:14:32.175190 20044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:14:32.175243 20044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:14:32.175263 20044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:14:32.175279 20044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:14:32.175295 20044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:14:32.180780 20044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:46409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:44427,127.17.44.253:45173,127.17.44.252:45509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:44427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:14:32.181818 20044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:14:32.182842 20044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:14:32.191818 20051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:32.191797 20054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:32.191797 20050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:32.192104 20044 server_base.cc:1034] running on GCE node
I20260812 11:14:32.192701 20044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:14:32.193863 20044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:14:32.195039 20044 hybrid_clock.cc:648] HybridClock initialized: now 1786533272195022 us; error 38 us; skew 500 ppm
Aug 12 11:14:32 dist-test-slave-drl0 krb5kdc[20022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533272, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:14:32.205165 20044 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:14:32.206115 20044 webserver.cc:460] Webserver started at http://127.17.44.254:36737/ using document root <none> and password file <none>
I20260812 11:14:32.206660 20044 fs_manager.cc:359] Metadata directory not provided
I20260812 11:14:32.206787 20044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:14:32.207125 20044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:14:32.209209 20044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "53ed5eda1e2d4f128b71158260a48f90"
format_stamp: "Formatted at 2026-08-12 11:14:32 on dist-test-slave-drl0"
server_key: "cbfdfd4dd30a72f5c731dc23a67feb99"
server_key_iv: "e75187eceadf4c1e1eab93442ddf2d44"
server_key_version: "encryptionkey@0"
I20260812 11:14:32.209798 20044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "53ed5eda1e2d4f128b71158260a48f90"
format_stamp: "Formatted at 2026-08-12 11:14:32 on dist-test-slave-drl0"
server_key: "cbfdfd4dd30a72f5c731dc23a67feb99"
server_key_iv: "e75187eceadf4c1e1eab93442ddf2d44"
server_key_version: "encryptionkey@0"
I20260812 11:14:32.213677 20044 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 11:14:32.217265 20064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:14:32.219192 20044 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 11:14:32.219396 20044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "53ed5eda1e2d4f128b71158260a48f90"
format_stamp: "Formatted at 2026-08-12 11:14:32 on dist-test-slave-drl0"
server_key: "cbfdfd4dd30a72f5c731dc23a67feb99"
server_key_iv: "e75187eceadf4c1e1eab93442ddf2d44"
server_key_version: "encryptionkey@0"
I20260812 11:14:32.219550 20044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:14:32.237471 20044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:14:32.240702 20044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:14:32.241030 20044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:14:32.248955 20044 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:44427
I20260812 11:14:32.249255 20153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:44427 every 8 connection(s)
I20260812 11:14:32.250052 20044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:14:32.252918 20154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:14:32.256829 20154 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44427 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:14:32.258008 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 20044
I20260812 11:14:32.258150 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:14:32.258424 17587 external_mini_cluster.cc:1428] Setting key e1d7d767f92058dfed1bf6098c55c1b3
Aug 12 11:14:32 dist-test-slave-drl0 krb5kdc[20022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533272, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.44.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.253@KRBTEST.COM" created.
I20260812 11:14:32.274227 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.259366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:44427 (local address 127.0.0.1:51930)
0812 11:14:32.260212 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.260362 (+ 150us) client_negotiation.cc:174] Beginning negotiation
0812 11:14:32.262089 (+ 1727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:14:32.263012 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:14:32.263017 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:14:32.263461 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:14:32.263704 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:32.263716 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.264995 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.264998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:32.265742 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:32.265751 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.266406 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.266410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:32.266501 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:32.267146 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:14:32.267163 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:14:32.268853 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:14:32.270965 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:32.270975 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:32.270985 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:32.271256 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:32.271603 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:32.271606 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:32.271608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:32.271755 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:32.272559 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:14:32.272568 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:14:32.272751 (+ 183us) client_negotiation.cc:769] Sending connection context
0812 11:14:32.273054 (+ 303us) client_negotiation.cc:240] Negotiation successful
0812 11:14:32.273423 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":131,"threads_started":1}
I20260812 11:14:32.274233 20157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.260080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51930 (local address 127.17.44.254:44427)
0812 11:14:32.260365 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:32.260377 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:32.260481 (+ 104us) server_negotiation.cc:407] Connection header received
0812 11:14:32.262475 (+ 1994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:32.262501 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:32.262759 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:32.263015 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:32.263860 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.264879 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.265871 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.266254 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.266433 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:32.269007 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:32.269013 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:32.269026 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:32.269055 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:32.270829 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.271374 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.271380 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.271382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.271513 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.271991 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.271995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.271996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.272409 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:32.272546 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:32.273175 (+ 629us) server_negotiation.cc:299] Negotiation successful
0812 11:14:32.273626 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":104,"threads_started":1}
I20260812 11:14:32.277257 20154 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:14:32.277997 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.277737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:45173 (local address 127.0.0.1:60224)
0812 11:14:32.277864 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.277942 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.253:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 11:14:32.278311 20067 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.44.253:45173: connect: Connection refused (error 111)
I20260812 11:14:32.279646 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.279411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:45173 (local address 127.0.0.1:60232)
0812 11:14:32.279551 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.279598 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.253:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260812 11:14:32.279984 20154 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.253:45173: Network error: Client connection negotiation failed: client connection to 127.17.44.253:45173: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.253.keytab.
Entry for principal kudu/127.17.44.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.253.keytab.
I20260812 11:14:32.289800 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.253:45173
--webserver_interface=127.17.44.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.253.keytab
--principal=kudu/127.17.44.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:46409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:44427,127.17.44.253:45173,127.17.44.252:45509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:14:32.329355 20154 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 } attempt: 1
I20260812 11:14:32.330529 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.330143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:45173 (local address 127.0.0.1:60242)
0812 11:14:32.330366 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.330430 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.253:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260812 11:14:32.331401 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.331186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:45173 (local address 127.0.0.1:60258)
0812 11:14:32.331297 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.331350 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.253:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 11:14:32.331533 20154 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.253:45173: Network error: Client connection negotiation failed: client connection to 127.17.44.253:45173: connect: Connection refused (error 111)
I20260812 11:14:32.399727 20154 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 } attempt: 2
I20260812 11:14:32.400638 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.400338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:45173 (local address 127.0.0.1:60260)
0812 11:14:32.400502 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.400564 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.253:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 11:14:32.401577 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.401254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:45173 (local address 127.0.0.1:60266)
0812 11:14:32.401462 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.401520 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.253:45173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260812 11:14:32.401716 20154 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.253:45173: Network error: Client connection negotiation failed: client connection to 127.17.44.253:45173: connect: Connection refused (error 111)
W20260812 11:14:32.411192 20162 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 11:14:32.411497 20162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:14:32.411562 20162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:14:32.411656 20162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:14:32.415709 20162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:14:32.415881 20162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:14:32.415954 20162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:14:32.415993 20162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:14:32.416028 20162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:14:32.422618 20162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:46409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:44427,127.17.44.253:45173,127.17.44.252:45509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.253.keytab
--principal=kudu/127.17.44.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.253:45173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.44.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:14:32.423848 20162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:14:32.424957 20162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:14:32.434254 20171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:32.434541 20169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:32.434254 20178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:32.434728 20162 server_base.cc:1034] running on GCE node
I20260812 11:14:32.435339 20162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:14:32.436427 20162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:14:32.437636 20162 hybrid_clock.cc:648] HybridClock initialized: now 1786533272437616 us; error 32 us; skew 500 ppm
Aug 12 11:14:32 dist-test-slave-drl0 krb5kdc[20022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533272, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:14:32.440982 20162 init.cc:377] Logged in from keytab as kudu/127.17.44.253@KRBTEST.COM (short username kudu)
I20260812 11:14:32.441879 20162 webserver.cc:460] Webserver started at http://127.17.44.253:36983/ using document root <none> and password file <none>
I20260812 11:14:32.442447 20162 fs_manager.cc:359] Metadata directory not provided
I20260812 11:14:32.442512 20162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:14:32.442679 20162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:14:32.444741 20162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-1/data/instance:
uuid: "a0158d9002234b0a9fb7b641b045fe4d"
format_stamp: "Formatted at 2026-08-12 11:14:32 on dist-test-slave-drl0"
server_key: "ac301df41989d5297cc3658650e4cf05"
server_key_iv: "f9d8d7af00cf45a7f9d57a61ae53e4a6"
server_key_version: "encryptionkey@0"
I20260812 11:14:32.445255 20162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-1/wal/instance:
uuid: "a0158d9002234b0a9fb7b641b045fe4d"
format_stamp: "Formatted at 2026-08-12 11:14:32 on dist-test-slave-drl0"
server_key: "ac301df41989d5297cc3658650e4cf05"
server_key_iv: "f9d8d7af00cf45a7f9d57a61ae53e4a6"
server_key_version: "encryptionkey@0"
I20260812 11:14:32.449501 20162 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 11:14:32.452219 20188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:14:32.453691 20162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 11:14:32.453949 20162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-1/wal
uuid: "a0158d9002234b0a9fb7b641b045fe4d"
format_stamp: "Formatted at 2026-08-12 11:14:32 on dist-test-slave-drl0"
server_key: "ac301df41989d5297cc3658650e4cf05"
server_key_iv: "f9d8d7af00cf45a7f9d57a61ae53e4a6"
server_key_version: "encryptionkey@0"
I20260812 11:14:32.454385 20162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:14:32.472364 20162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:14:32.475687 20162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:14:32.475900 20162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:14:32.483340 20162 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.253:45173
I20260812 11:14:32.483656 20270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.253:45173 every 8 connection(s)
I20260812 11:14:32.484308 20162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-1/data/info.pb
I20260812 11:14:32.487272 20272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:14:32.487303 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 20162
I20260812 11:14:32.487430 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-1/wal/instance
I20260812 11:14:32.487744 17587 external_mini_cluster.cc:1428] Setting key 861a37de33a3ff0356e94fac7acee52f
I20260812 11:14:32.491829 20272 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44427 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:14:32.492959 20154 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 } attempt: 3
WARNING: no policy specified for kudu/127.17.44.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.252@KRBTEST.COM" created.
Aug 12 11:14:32 dist-test-slave-drl0 krb5kdc[20022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533272, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.253@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
Aug 12 11:14:32 dist-test-slave-drl0 krb5kdc[20022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533272, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for kudu/127.17.44.253@KRBTEST.COM
I20260812 11:14:32.508030 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.493532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:45173 (local address 127.0.0.1:60280)
0812 11:14:32.493742 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.493757 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:14:32.493851 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:14:32.496350 (+ 2499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:14:32.496354 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:14:32.496574 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:14:32.496724 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:32.496734 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.498798 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.498803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:32.499561 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:32.499573 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.500312 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.500317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:32.500499 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:32.501072 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:14:32.501092 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:14:32.503126 (+ 2034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:14:32.505297 (+ 2171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:32.505301 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:32.505303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:32.505503 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:32.505907 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:32.505913 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:32.505918 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:32.506028 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:32.507453 (+ 1425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:14:32.507456 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:14:32.507614 (+ 158us) client_negotiation.cc:769] Sending connection context
0812 11:14:32.507731 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 11:14:32.507898 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"spinlock_wait_cycles":10624}
I20260812 11:14:32.509042 20276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.494078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:44427 (local address 127.0.0.1:51934)
0812 11:14:32.494657 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.494869 (+ 212us) client_negotiation.cc:174] Beginning negotiation
0812 11:14:32.495910 (+ 1041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:14:32.496343 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:14:32.496352 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:14:32.496691 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:14:32.497152 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:32.497163 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.497928 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.497934 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:32.499159 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:32.499169 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.499688 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.499693 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:32.499973 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:32.500707 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:14:32.500733 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:14:32.502812 (+ 2079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:14:32.505202 (+ 2390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:32.505215 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:32.505228 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:32.505550 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:32.506164 (+ 614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:32.506170 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:32.506175 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:32.506331 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:32.507085 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:14:32.507094 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:14:32.507310 (+ 216us) client_negotiation.cc:769] Sending connection context
0812 11:14:32.507548 (+ 238us) client_negotiation.cc:240] Negotiation successful
0812 11:14:32.507921 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":419,"spinlock_wait_cycles":5376,"thread_start_us":140,"threads_started":1}
I20260812 11:14:32.509092 20277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.494079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60280 (local address 127.17.44.253:45173)
0812 11:14:32.494671 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:32.494691 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:32.494741 (+ 50us) server_negotiation.cc:407] Connection header received
0812 11:14:32.495836 (+ 1095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:32.495865 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:32.496123 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:32.496897 (+ 774us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:32.497044 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.498649 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.499688 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.500087 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.500226 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:32.503277 (+ 3051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:32.503284 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:32.503296 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:32.503349 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:32.505141 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.505616 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.505622 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.505624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.505790 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.506287 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.506293 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.506294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.507311 (+ 1017us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:32.507466 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:32.508388 (+ 922us) server_negotiation.cc:299] Negotiation successful
0812 11:14:32.508549 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":458,"spinlock_wait_cycles":4992,"thread_start_us":156,"threads_started":1}
I20260812 11:14:32.510391 20157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.493816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51934 (local address 127.17.44.254:44427)
0812 11:14:32.493925 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:32.493930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:32.495011 (+ 1081us) server_negotiation.cc:407] Connection header received
0812 11:14:32.496119 (+ 1108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:32.496122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:32.496186 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:32.496277 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:32.497294 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.497808 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.499285 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.499533 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.499649 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:32.503036 (+ 3387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:32.503041 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:32.503045 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:32.503079 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:32.505078 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.505710 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.505714 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.505715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.505801 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.506570 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.506575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.506577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.506874 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:32.507153 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:32.510015 (+ 2862us) server_negotiation.cc:299] Negotiation successful
0812 11:14:32.510229 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 11:14:32.511695 20272 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:14:32.512207 20154 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:14:32.512913 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.512642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:45509 (local address 127.0.0.1:60246)
0812 11:14:32.512754 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.512816 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:45509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 11:14:32.513675 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.513434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:45509 (local address 127.0.0.1:60248)
0812 11:14:32.513572 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.513620 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:45509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260812 11:14:32.513825 20154 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.252:45509: Network error: Client connection negotiation failed: client connection to 127.17.44.252:45509: connect: Connection refused (error 111)
Aug 12 11:14:32 dist-test-slave-drl0 krb5kdc[20022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533272, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.253@KRBTEST.COM for kudu/127.17.44.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.252.keytab.
Entry for principal kudu/127.17.44.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.252.keytab.
I20260812 11:14:32.518559 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.252:45509
--webserver_interface=127.17.44.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.252.keytab
--principal=kudu/127.17.44.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:46409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:44427,127.17.44.253:45173,127.17.44.252:45509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:14:32.521934 20276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.512297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:45173 (local address 127.0.0.1:60296)
0812 11:14:32.512397 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.512411 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:14:32.512490 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:14:32.512758 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:14:32.512763 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:14:32.512928 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:14:32.513083 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:32.513132 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.514075 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.514078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:32.514686 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:32.514695 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.515271 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.515274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:32.515371 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:32.515816 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:14:32.515828 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:14:32.517189 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:14:32.519797 (+ 2608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:32.519803 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:32.519808 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:32.520098 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:32.520434 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:32.520437 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:32.520526 (+ 89us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:32.520720 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:32.521390 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:14:32.521392 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:14:32.521477 (+ 85us) client_negotiation.cc:769] Sending connection context
0812 11:14:32.521574 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 11:14:32.521675 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 11:14:32.522047 20277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.512371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60296 (local address 127.17.44.253:45173)
0812 11:14:32.512468 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:32.512472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:32.512484 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:14:32.512609 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:32.512612 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:32.512660 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:32.512748 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:32.513254 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.513975 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.514821 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.515167 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.515254 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:32.517350 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:32.517354 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:32.517357 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:32.517382 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:32.519143 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.520226 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.520229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.520230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.520321 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.520903 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.520906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.520907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.521220 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:32.521352 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:32.521660 (+ 308us) server_negotiation.cc:299] Negotiation successful
0812 11:14:32.521793 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 11:14:32.523121 20272 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:14:32.523819 20276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.523553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:45509 (local address 127.0.0.1:60258)
0812 11:14:32.523689 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.523770 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:45509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260812 11:14:32.524041 20192 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.44.252:45509: connect: Connection refused (error 111)
I20260812 11:14:32.524947 20276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.524757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:45509 (local address 127.0.0.1:60270)
0812 11:14:32.524859 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.524902 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:45509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 11:14:32.525198 20272 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.252:45509: Network error: Client connection negotiation failed: client connection to 127.17.44.252:45509: connect: Connection refused (error 111)
I20260812 11:14:32.545084 20154 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } attempt: 1
I20260812 11:14:32.545886 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.545607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:45509 (local address 127.0.0.1:60278)
0812 11:14:32.545752 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.545816 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:45509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 11:14:32.546571 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.546368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:45509 (local address 127.0.0.1:60286)
0812 11:14:32.546476 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.546525 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:45509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 11:14:32.546926 20154 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.252:45509: Network error: Client connection negotiation failed: client connection to 127.17.44.252:45509: connect: Connection refused (error 111)
I20260812 11:14:32.574518 20272 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } attempt: 1
I20260812 11:14:32.575474 20276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.575185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:45509 (local address 127.0.0.1:60294)
0812 11:14:32.575332 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.575402 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:45509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 11:14:32.576251 20276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.576039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:45509 (local address 127.0.0.1:60304)
0812 11:14:32.576149 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.576194 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:45509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 11:14:32.576439 20272 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.252:45509: Network error: Client connection negotiation failed: client connection to 127.17.44.252:45509: connect: Connection refused (error 111)
I20260812 11:14:32.622222 20154 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } attempt: 2
I20260812 11:14:32.623126 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.622793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:45509 (local address 127.0.0.1:60308)
0812 11:14:32.622980 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.623048 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:45509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 11:14:32.623939 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.623728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:45509 (local address 127.0.0.1:60316)
0812 11:14:32.623837 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.623890 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:45509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 11:14:32.624117 20154 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.252:45509: Network error: Client connection negotiation failed: client connection to 127.17.44.252:45509: connect: Connection refused (error 111)
W20260812 11:14:32.642186 20280 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 11:14:32.642453 20280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:14:32.642477 20280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:14:32.642515 20280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 11:14:32.644637 20272 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } attempt: 2
I20260812 11:14:32.645519 20276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.645197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:45509 (local address 127.0.0.1:60332)
0812 11:14:32.645385 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.645446 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:45509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260812 11:14:32.646799 20276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.646443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:45509 (local address 127.0.0.1:60338)
0812 11:14:32.646681 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.646734 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:45509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260812 11:14:32.647150 20272 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.252:45509: Network error: Client connection negotiation failed: client connection to 127.17.44.252:45509: connect: Connection refused (error 111)
W20260812 11:14:32.647845 20280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:14:32.647902 20280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:14:32.647922 20280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:14:32.647939 20280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:14:32.647984 20280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:14:32.653880 20280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:46409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:44427,127.17.44.253:45173,127.17.44.252:45509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.252.keytab
--principal=kudu/127.17.44.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.252:45509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.44.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:14:32.654904 20280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:14:32.655949 20280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:14:32.665103 20290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:32.665130 20291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:32.665186 20280 server_base.cc:1034] running on GCE node
W20260812 11:14:32.665127 20293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:32.665989 20280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:14:32.667115 20280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:14:32.668269 20280 hybrid_clock.cc:648] HybridClock initialized: now 1786533272668255 us; error 27 us; skew 500 ppm
Aug 12 11:14:32 dist-test-slave-drl0 krb5kdc[20022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533272, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:14:32.671177 20280 init.cc:377] Logged in from keytab as kudu/127.17.44.252@KRBTEST.COM (short username kudu)
I20260812 11:14:32.672106 20280 webserver.cc:460] Webserver started at http://127.17.44.252:36583/ using document root <none> and password file <none>
I20260812 11:14:32.672622 20280 fs_manager.cc:359] Metadata directory not provided
I20260812 11:14:32.672683 20280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:14:32.672853 20280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:14:32.674784 20280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-2/data/instance:
uuid: "0a104fc4fb1c421eb7352788001778df"
format_stamp: "Formatted at 2026-08-12 11:14:32 on dist-test-slave-drl0"
server_key: "3c1e78c07b3e1922f1dbcea4d41bafea"
server_key_iv: "d7f48ed2451591c5c835b48c76d88c1f"
server_key_version: "encryptionkey@0"
I20260812 11:14:32.675387 20280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-2/wal/instance:
uuid: "0a104fc4fb1c421eb7352788001778df"
format_stamp: "Formatted at 2026-08-12 11:14:32 on dist-test-slave-drl0"
server_key: "3c1e78c07b3e1922f1dbcea4d41bafea"
server_key_iv: "d7f48ed2451591c5c835b48c76d88c1f"
server_key_version: "encryptionkey@0"
I20260812 11:14:32.679068 20280 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 11:14:32.681735 20304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:14:32.683588 20280 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 11:14:32.683694 20280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-2/wal
uuid: "0a104fc4fb1c421eb7352788001778df"
format_stamp: "Formatted at 2026-08-12 11:14:32 on dist-test-slave-drl0"
server_key: "3c1e78c07b3e1922f1dbcea4d41bafea"
server_key_iv: "d7f48ed2451591c5c835b48c76d88c1f"
server_key_version: "encryptionkey@0"
I20260812 11:14:32.683779 20280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:14:32.701969 20280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:14:32.705713 20280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:14:32.705876 20280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:14:32.714771 20280 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.252:45509
I20260812 11:14:32.715118 20399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.252:45509 every 8 connection(s)
I20260812 11:14:32.715786 20280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-2/data/info.pb
I20260812 11:14:32.717746 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 20280
I20260812 11:14:32.717896 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786532961723938-17587-0/minicluster-data/master-2/wal/instance
I20260812 11:14:32.718185 17587 external_mini_cluster.cc:1428] Setting key 163452ea51143308dbf1e48efe3185c0
I20260812 11:14:32.718863 20400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:14:32.723366 20154 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } attempt: 3
I20260812 11:14:32.723949 20400 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44427 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 11:14:32 dist-test-slave-drl0 krb5kdc[20022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.252@KRBTEST.COM
Aug 12 11:14:32 dist-test-slave-drl0 krb5kdc[20022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533272, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for kudu/127.17.44.252@KRBTEST.COM
Aug 12 11:14:32 dist-test-slave-drl0 krb5kdc[20022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533272, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.252@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:14:32.736043 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.723934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:45509 (local address 127.0.0.1:60348)
0812 11:14:32.724295 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.724310 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:14:32.724398 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:14:32.724918 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:14:32.724921 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:14:32.725149 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:14:32.725258 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:32.725265 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.726225 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.726228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:32.727012 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:32.727022 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.728045 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.728052 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:32.728245 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:32.728777 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:14:32.728791 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:14:32.730323 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:14:32.733726 (+ 3403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:32.733730 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:32.733731 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:32.733987 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:32.734495 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:32.734500 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:32.734502 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:32.734596 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:32.735498 (+ 902us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:14:32.735501 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:14:32.735601 (+ 100us) client_negotiation.cc:769] Sending connection context
0812 11:14:32.735729 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 11:14:32.735867 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":32}
I20260812 11:14:32.737146 20410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.724041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60348 (local address 127.17.44.252:45509)
0812 11:14:32.724670 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:32.724678 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:32.724692 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:14:32.724730 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:32.724733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:32.724770 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:32.724856 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:32.725401 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.726091 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.727144 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.727908 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.728033 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:32.730473 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:32.730481 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:32.730493 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:32.730523 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:32.733578 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.734214 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.734223 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.734233 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.734383 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.734699 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.734702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.734704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.735380 (+ 676us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:32.735533 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:32.735876 (+ 343us) server_negotiation.cc:299] Negotiation successful
0812 11:14:32.736217 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":394,"server-negotiator.queue_time_us":559,"thread_start_us":127,"threads_started":1}
I20260812 11:14:32.737231 20408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.719839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60344 (local address 127.17.44.252:45509)
0812 11:14:32.720622 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:32.720642 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:32.720682 (+ 40us) server_negotiation.cc:407] Connection header received
0812 11:14:32.721475 (+ 793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:32.721494 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:32.721806 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:32.722082 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:32.723161 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.725109 (+ 1948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.726283 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.727021 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.727345 (+ 324us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:32.731031 (+ 3686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:32.731034 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:32.731037 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:32.731088 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:32.733754 (+ 2666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.734616 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.734620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.734621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.734722 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.735226 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.735231 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.735234 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.736096 (+ 862us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:32.736872 (+ 776us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:32.736944 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 11:14:32.737052 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":480,"spinlock_wait_cycles":8704,"thread_start_us":291,"threads_started":1}
I20260812 11:14:32.737931 20157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.726035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51938 (local address 127.17.44.254:44427)
0812 11:14:32.726157 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:32.726162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:32.726536 (+ 374us) server_negotiation.cc:407] Connection header received
0812 11:14:32.726721 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:32.726725 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:32.726769 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:32.726851 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:32.728236 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.728871 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.729995 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.730244 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.730346 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:32.733959 (+ 3613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:32.733962 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:32.733965 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:32.733986 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:32.735781 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.736414 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.736420 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.736425 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.736511 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.736847 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.736851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.736852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.737143 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:32.737217 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:32.737677 (+ 460us) server_negotiation.cc:299] Negotiation successful
0812 11:14:32.737777 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":44}
I20260812 11:14:32.737953 20411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.725981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:44427 (local address 127.0.0.1:51938)
0812 11:14:32.726414 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.726464 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 11:14:32.726616 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:14:32.726916 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:14:32.726921 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:14:32.727389 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:14:32.728061 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:32.728069 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.729174 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.729180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:32.729872 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:32.729882 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.730409 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.730422 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:32.730511 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:32.731038 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:14:32.731073 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:14:32.733834 (+ 2761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:14:32.735918 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:32.735928 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:32.735934 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:32.736286 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:32.736633 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:32.736636 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:32.736638 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:32.736729 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:32.737304 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:14:32.737309 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:14:32.737439 (+ 130us) client_negotiation.cc:769] Sending connection context
0812 11:14:32.737606 (+ 167us) client_negotiation.cc:240] Negotiation successful
0812 11:14:32.737780 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":437,"spinlock_wait_cycles":16128,"thread_start_us":100,"threads_started":1}
I20260812 11:14:32.738353 20272 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } attempt: 3
I20260812 11:14:32.739140 20400 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:14:32.743620 20154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90: Bootstrap starting.
Aug 12 11:14:32 dist-test-slave-drl0 krb5kdc[20022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533272, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.253@KRBTEST.COM for kudu/127.17.44.252@KRBTEST.COM
Aug 12 11:14:32 dist-test-slave-drl0 krb5kdc[20022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533272, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.252@KRBTEST.COM for kudu/127.17.44.253@KRBTEST.COM
I20260812 11:14:32.747181 20154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90: Neither blocks nor log segments found. Creating new log.
I20260812 11:14:32.748236 20154 log.cc:824] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90: Log is configured to *not* fsync() on all Append() calls
Aug 12 11:14:32 dist-test-slave-drl0 krb5kdc[20022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:14:32.749773 20276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.738819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:45509 (local address 127.0.0.1:60356)
0812 11:14:32.739017 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.739034 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 11:14:32.739126 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:14:32.739965 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:14:32.739968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:14:32.740132 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:14:32.740212 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:32.740217 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.741283 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.741287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:32.742084 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:32.742093 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.742776 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.742783 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:32.742854 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:32.743551 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:14:32.743566 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:14:32.745121 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:14:32.748005 (+ 2884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:32.748008 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:32.748011 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:32.748253 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:32.748581 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:32.748584 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:32.748587 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:32.748662 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:32.749115 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:14:32.749118 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:14:32.749194 (+ 76us) client_negotiation.cc:769] Sending connection context
0812 11:14:32.749461 (+ 267us) client_negotiation.cc:240] Negotiation successful
0812 11:14:32.749601 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 11:14:32.750044 20408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.739481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60356 (local address 127.17.44.252:45509)
0812 11:14:32.739753 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:32.739756 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:32.739768 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:14:32.739796 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:32.739801 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:32.739855 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:32.739926 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:32.740359 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.741164 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.742215 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.742657 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.745463 (+ 2806us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:32.745568 (+ 105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:32.745574 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:32.745580 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:32.745601 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:32.747870 (+ 2269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.748391 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.748394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.748396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.748481 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.748770 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.748773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.748774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.749005 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:32.749253 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:32.749514 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 11:14:32.749618 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":205,"spinlock_wait_cycles":4352}
I20260812 11:14:32.750591 20154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90: No bootstrap required, opened a new log
I20260812 11:14:32.750682 20277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.739696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60306 (local address 127.17.44.253:45173)
0812 11:14:32.739801 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:32.739805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:32.739818 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:14:32.739854 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:32.739857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:32.740155 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:32.740276 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:32.740735 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.741437 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.742431 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.742750 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.742815 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:32.745839 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:32.745845 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:32.745848 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:32.745875 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:32.748023 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.748546 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.748551 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.748556 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.748637 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.748977 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.748980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.748982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.749285 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:32.749357 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:32.750387 (+ 1030us) server_negotiation.cc:299] Negotiation successful
0812 11:14:32.750508 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":43}
I20260812 11:14:32.751142 20411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.739588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:45173 (local address 127.0.0.1:60306)
0812 11:14:32.739689 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.739704 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:14:32.739777 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:14:32.740306 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:14:32.740308 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:14:32.740472 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:14:32.740605 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:32.740614 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.741564 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.741567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:32.742319 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:32.742326 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.743006 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.743012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:32.743118 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:32.743551 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:14:32.743566 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:14:32.745665 (+ 2099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:14:32.748132 (+ 2467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:32.748136 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:32.748140 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:32.748431 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:32.748754 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:32.748759 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:32.748761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:32.748865 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:32.750078 (+ 1213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:14:32.750084 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:14:32.750193 (+ 109us) client_negotiation.cc:769] Sending connection context
0812 11:14:32.750798 (+ 605us) client_negotiation.cc:240] Negotiation successful
0812 11:14:32.750979 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":25,"spinlock_wait_cycles":2816}
I20260812 11:14:32.753607 20272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0158d9002234b0a9fb7b641b045fe4d: Bootstrap starting.
I20260812 11:14:32.754659 20154 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ed5eda1e2d4f128b71158260a48f90" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44427 } } peers { permanent_uuid: "a0158d9002234b0a9fb7b641b045fe4d" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 } } peers { permanent_uuid: "0a104fc4fb1c421eb7352788001778df" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } }
I20260812 11:14:32.755121 20154 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ed5eda1e2d4f128b71158260a48f90, State: Initialized, Role: FOLLOWER
I20260812 11:14:32.755507 20154 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ed5eda1e2d4f128b71158260a48f90" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44427 } } peers { permanent_uuid: "a0158d9002234b0a9fb7b641b045fe4d" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 } } peers { permanent_uuid: "0a104fc4fb1c421eb7352788001778df" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } }
I20260812 11:14:32.756062 20413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ed5eda1e2d4f128b71158260a48f90" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44427 } } peers { permanent_uuid: "a0158d9002234b0a9fb7b641b045fe4d" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 } } peers { permanent_uuid: "0a104fc4fb1c421eb7352788001778df" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } } }
I20260812 11:14:32.756208 20413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:14:32.756624 20272 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0158d9002234b0a9fb7b641b045fe4d: Neither blocks nor log segments found. Creating new log.
I20260812 11:14:32.757038 20154 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:14:32.757588 20272 log.cc:824] T 00000000000000000000000000000000 P a0158d9002234b0a9fb7b641b045fe4d: Log is configured to *not* fsync() on all Append() calls
I20260812 11:14:32.758648 20400 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:14:32.759972 20272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0158d9002234b0a9fb7b641b045fe4d: No bootstrap required, opened a new log
I20260812 11:14:32.761308 20157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.741174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51950 (local address 127.17.44.254:44427)
0812 11:14:32.741988 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:32.741993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:32.742004 (+ 11us) server_negotiation.cc:407] Connection header received
0812 11:14:32.742034 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:32.742036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:32.742076 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:32.742145 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:32.743037 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.743835 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.745195 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.745542 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.745645 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:32.749525 (+ 3880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:32.749528 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:32.749530 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:32.749554 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:32.758617 (+ 9063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.759296 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.759302 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.759306 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.759412 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.759779 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.759782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.759783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.760182 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:32.760371 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:32.761036 (+ 665us) server_negotiation.cc:299] Negotiation successful
0812 11:14:32.761148 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":744}
Aug 12 11:14:32 dist-test-slave-drl0 krb5kdc[20022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533272, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.252@KRBTEST.COM for kudu/127.17.44.252@KRBTEST.COM
I20260812 11:14:32.764523 20272 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0158d9002234b0a9fb7b641b045fe4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ed5eda1e2d4f128b71158260a48f90" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44427 } } peers { permanent_uuid: "a0158d9002234b0a9fb7b641b045fe4d" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 } } peers { permanent_uuid: "0a104fc4fb1c421eb7352788001778df" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } }
I20260812 11:14:32.764832 20272 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0158d9002234b0a9fb7b641b045fe4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0158d9002234b0a9fb7b641b045fe4d, State: Initialized, Role: FOLLOWER
I20260812 11:14:32.765863 20272 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0158d9002234b0a9fb7b641b045fe4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ed5eda1e2d4f128b71158260a48f90" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44427 } } peers { permanent_uuid: "a0158d9002234b0a9fb7b641b045fe4d" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 } } peers { permanent_uuid: "0a104fc4fb1c421eb7352788001778df" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } }
W20260812 11:14:32.766278 20435 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:14:32.766412 20435 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:14:32.762598 20044 master_runner.cc:186] Error getting master registration for 127.17.44.253:45173: OK, instance_id { permanent_uuid: "a0158d9002234b0a9fb7b641b045fe4d" instance_seqno: 1786533272481165 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 11:14:32.766983 20044 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 11:14:32.767536 20272 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0158d9002234b0a9fb7b641b045fe4d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:14:32.768180 20411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.759189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:45509 (local address 127.0.0.1:60358)
0812 11:14:32.759422 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:32.759436 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:14:32.759524 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:14:32.759818 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:14:32.759821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:14:32.760002 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:14:32.760096 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:32.760103 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.761044 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.761047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:32.761821 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:32.761841 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.762455 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.762458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:32.762529 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:32.763035 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:14:32.763049 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:14:32.764515 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:14:32.766509 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:32.766511 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:32.766513 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:32.766835 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:32.767183 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:32.767185 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:32.767187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:32.767271 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:32.767684 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:14:32.767686 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:14:32.767784 (+ 98us) client_negotiation.cc:769] Sending connection context
0812 11:14:32.767885 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 11:14:32.768022 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164}
I20260812 11:14:32.768697 20408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.759407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60358 (local address 127.17.44.252:45509)
0812 11:14:32.759518 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:32.759522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:32.759536 (+ 14us) server_negotiation.cc:407] Connection header received
0812 11:14:32.759620 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:32.759623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:32.759664 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:32.759725 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:32.760209 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.760907 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.761957 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.762265 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.762360 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:32.764652 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:32.764656 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:32.764658 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:32.764680 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:32.766399 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.766992 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.766995 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.766996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.767081 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.767364 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.767367 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.767368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.767556 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:32.767651 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:32.768432 (+ 781us) server_negotiation.cc:299] Negotiation successful
0812 11:14:32.768539 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:14:32.771781 20400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a104fc4fb1c421eb7352788001778df: Bootstrap starting.
Aug 12 11:14:32 dist-test-slave-drl0 krb5kdc[20022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.253@KRBTEST.COM
I20260812 11:14:32.774410 20400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a104fc4fb1c421eb7352788001778df: Neither blocks nor log segments found. Creating new log.
I20260812 11:14:32.775306 20400 log.cc:824] T 00000000000000000000000000000000 P 0a104fc4fb1c421eb7352788001778df: Log is configured to *not* fsync() on all Append() calls
I20260812 11:14:32.777659 20400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a104fc4fb1c421eb7352788001778df: No bootstrap required, opened a new log
I20260812 11:14:32.775285 20162 master_runner.cc:186] Error getting master registration for 127.17.44.252:45509: OK, instance_id { permanent_uuid: "0a104fc4fb1c421eb7352788001778df" instance_seqno: 1786533272712814 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 11:14:32.779433 20162 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 11:14:32.781333 20277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.763034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60320 (local address 127.17.44.253:45173)
0812 11:14:32.763151 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:32.763155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:32.763167 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:14:32.763201 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:32.763204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:32.763243 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:32.763316 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:32.767812 (+ 4496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.768599 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.769669 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.770017 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.770094 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:32.773101 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:32.773105 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:32.773108 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:32.773136 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:32.774863 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.779673 (+ 4810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.779677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.779679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.779770 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.780093 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.780099 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.780103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.780341 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:32.780500 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:32.780863 (+ 363us) server_negotiation.cc:299] Negotiation successful
0812 11:14:32.781008 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 11:14:32.781862 20400 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a104fc4fb1c421eb7352788001778df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ed5eda1e2d4f128b71158260a48f90" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44427 } } peers { permanent_uuid: "a0158d9002234b0a9fb7b641b045fe4d" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 } } peers { permanent_uuid: "0a104fc4fb1c421eb7352788001778df" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } }
I20260812 11:14:32.782157 20400 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a104fc4fb1c421eb7352788001778df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a104fc4fb1c421eb7352788001778df, State: Initialized, Role: FOLLOWER
I20260812 11:14:32.782536 20400 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a104fc4fb1c421eb7352788001778df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ed5eda1e2d4f128b71158260a48f90" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44427 } } peers { permanent_uuid: "a0158d9002234b0a9fb7b641b045fe4d" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 } } peers { permanent_uuid: "0a104fc4fb1c421eb7352788001778df" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } }
W20260812 11:14:32.783576 20458 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a0158d9002234b0a9fb7b641b045fe4d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260812 11:14:32.783910 20400 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a104fc4fb1c421eb7352788001778df [sys.catalog]: configured and running, proceeding with master startup.
W20260812 11:14:32.783689 20458 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:14:32.784848 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:14:32.786288 20117 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "53ed5eda1e2d4f128b71158260a48f90"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51950
I20260812 11:14:32.786593 20117 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 11:14:32.786726 20117 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:14:32.787192 20439 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0158d9002234b0a9fb7b641b045fe4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ed5eda1e2d4f128b71158260a48f90" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44427 } } peers { permanent_uuid: "a0158d9002234b0a9fb7b641b045fe4d" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 } } peers { permanent_uuid: "0a104fc4fb1c421eb7352788001778df" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } } }
I20260812 11:14:32.787312 20439 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0158d9002234b0a9fb7b641b045fe4d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:14:32.788306 20117 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ed5eda1e2d4f128b71158260a48f90" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44427 } } peers { permanent_uuid: "a0158d9002234b0a9fb7b641b045fe4d" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 } } peers { permanent_uuid: "0a104fc4fb1c421eb7352788001778df" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } }
I20260812 11:14:32.789459 20117 leader_election.cc:288] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90 [CANDIDATE]: Term 1 election: Requested vote from peers a0158d9002234b0a9fb7b641b045fe4d (127.17.44.253:45173), 0a104fc4fb1c421eb7352788001778df (127.17.44.252:45509)
I20260812 11:14:32.790196 20238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "53ed5eda1e2d4f128b71158260a48f90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a0158d9002234b0a9fb7b641b045fe4d"
I20260812 11:14:32.790426 20238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0158d9002234b0a9fb7b641b045fe4d [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:14:32.791721 20238 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a0158d9002234b0a9fb7b641b045fe4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ed5eda1e2d4f128b71158260a48f90 in term 1.
I20260812 11:14:32.792176 20067 leader_election.cc:302] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ed5eda1e2d4f128b71158260a48f90, a0158d9002234b0a9fb7b641b045fe4d; no voters:
I20260812 11:14:32.792445 20413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:14:32.792651 20413 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90 [term 1 LEADER]: Becoming Leader. State: Replica: 53ed5eda1e2d4f128b71158260a48f90, State: Running, Role: LEADER
I20260812 11:14:32.790794 20357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "53ed5eda1e2d4f128b71158260a48f90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0a104fc4fb1c421eb7352788001778df"
I20260812 11:14:32.792965 20413 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ed5eda1e2d4f128b71158260a48f90" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44427 } } peers { permanent_uuid: "a0158d9002234b0a9fb7b641b045fe4d" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 } } peers { permanent_uuid: "0a104fc4fb1c421eb7352788001778df" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } }
I20260812 11:14:32.793120 20357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a104fc4fb1c421eb7352788001778df [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:14:32.794407 20357 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0a104fc4fb1c421eb7352788001778df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ed5eda1e2d4f128b71158260a48f90 in term 1.
I20260812 11:14:32.794636 20413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53ed5eda1e2d4f128b71158260a48f90. Latest consensus state: current_term: 1 leader_uuid: "53ed5eda1e2d4f128b71158260a48f90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ed5eda1e2d4f128b71158260a48f90" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44427 } } peers { permanent_uuid: "a0158d9002234b0a9fb7b641b045fe4d" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 } } peers { permanent_uuid: "0a104fc4fb1c421eb7352788001778df" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } } }
I20260812 11:14:32.794741 20413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:32.786118 20463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a104fc4fb1c421eb7352788001778df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ed5eda1e2d4f128b71158260a48f90" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44427 } } peers { permanent_uuid: "a0158d9002234b0a9fb7b641b045fe4d" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 } } peers { permanent_uuid: "0a104fc4fb1c421eb7352788001778df" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } } }
I20260812 11:14:32.797905 20463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a104fc4fb1c421eb7352788001778df [sys.catalog]: This master's current role is: FOLLOWER
W20260812 11:14:32.797943 20480 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a104fc4fb1c421eb7352788001778df: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:14:32.798023 20480 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:14:32.799144 20484 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:14:32.801066 20280 master_runner.cc:263] No leader master found from master 0a104fc4fb1c421eb7352788001778df
I20260812 11:14:32.801128 20280 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 11:14:32.802253 20484 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:14:32.815651 20277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.798318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60322 (local address 127.17.44.253:45173)
0812 11:14:32.798414 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:32.798418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:32.802567 (+ 4149us) server_negotiation.cc:407] Connection header received
0812 11:14:32.802724 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:32.802727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:32.802774 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:32.802843 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:32.806220 (+ 3377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.807286 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.808625 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.808970 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.809047 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:32.811149 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:32.811153 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:32.811156 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:32.811184 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:32.812951 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.813858 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.813862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.813864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.813961 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.814323 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.814326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.814328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.814539 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:32.814650 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:32.815062 (+ 412us) server_negotiation.cc:299] Negotiation successful
0812 11:14:32.815197 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 11:14:32.819291 20157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.797178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51954 (local address 127.17.44.254:44427)
0812 11:14:32.797283 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:32.797288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:32.798776 (+ 1488us) server_negotiation.cc:407] Connection header received
0812 11:14:32.802735 (+ 3959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:32.802738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:32.802781 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:32.802830 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:32.803940 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.804658 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.805941 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.806576 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.806666 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:32.815035 (+ 8369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:32.815038 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:32.815042 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:32.815069 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:32.817436 (+ 2367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.817920 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.817924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.817925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.818011 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.818323 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.818327 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.818329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.818525 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:32.818629 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:32.818990 (+ 361us) server_negotiation.cc:299] Negotiation successful
0812 11:14:32.819118 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 11:14:32.825497 20408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:32.798799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60368 (local address 127.17.44.252:45509)
0812 11:14:32.798912 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:32.798916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:32.808710 (+ 9794us) server_negotiation.cc:407] Connection header received
0812 11:14:32.808864 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:32.808867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:32.808913 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:32.808963 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:32.815405 (+ 6442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.816157 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.818793 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:32.819136 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:32.819213 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:32.820926 (+ 1713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:32.820932 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:32.820934 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:32.820978 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:32.822702 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.823275 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.823279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.823280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.823385 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:32.823722 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:32.823725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:32.823727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:32.824254 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:32.824464 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:32.825058 (+ 594us) server_negotiation.cc:299] Negotiation successful
0812 11:14:32.825340 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 11:14:32.836390 20357 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0a104fc4fb1c421eb7352788001778df [term 1 FOLLOWER]: Refusing update from remote peer 53ed5eda1e2d4f128b71158260a48f90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:14:32.837247 20413 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a104fc4fb1c421eb7352788001778df" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:14:32.838557 20238 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a0158d9002234b0a9fb7b641b045fe4d [term 1 FOLLOWER]: Refusing update from remote peer 53ed5eda1e2d4f128b71158260a48f90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:14:32.842893 20463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a104fc4fb1c421eb7352788001778df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53ed5eda1e2d4f128b71158260a48f90. Latest consensus state: current_term: 1 leader_uuid: "53ed5eda1e2d4f128b71158260a48f90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ed5eda1e2d4f128b71158260a48f90" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44427 } } peers { permanent_uuid: "a0158d9002234b0a9fb7b641b045fe4d" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 } } peers { permanent_uuid: "0a104fc4fb1c421eb7352788001778df" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } } }
I20260812 11:14:32.843045 20463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a104fc4fb1c421eb7352788001778df [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:14:32.844595 20482 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0158d9002234b0a9fb7b641b045fe4d" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:14:32.848623 20439 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0158d9002234b0a9fb7b641b045fe4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53ed5eda1e2d4f128b71158260a48f90. Latest consensus state: current_term: 1 leader_uuid: "53ed5eda1e2d4f128b71158260a48f90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ed5eda1e2d4f128b71158260a48f90" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44427 } } peers { permanent_uuid: "a0158d9002234b0a9fb7b641b045fe4d" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 } } peers { permanent_uuid: "0a104fc4fb1c421eb7352788001778df" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } } }
I20260812 11:14:32.848747 20439 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0158d9002234b0a9fb7b641b045fe4d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:14:32.854851 20482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "53ed5eda1e2d4f128b71158260a48f90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ed5eda1e2d4f128b71158260a48f90" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44427 } } peers { permanent_uuid: "a0158d9002234b0a9fb7b641b045fe4d" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 } } peers { permanent_uuid: "0a104fc4fb1c421eb7352788001778df" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } } }
I20260812 11:14:32.855013 20482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:32.855130 20501 mvcc.cc:205] Tried to move back new op lower bound from 7317640285485150208 to 7317640285363122176. Current Snapshot: MvccSnapshot[applied={T|T < 7317640285485150208}]
I20260812 11:14:32.856499 20463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a104fc4fb1c421eb7352788001778df [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "53ed5eda1e2d4f128b71158260a48f90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ed5eda1e2d4f128b71158260a48f90" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44427 } } peers { permanent_uuid: "a0158d9002234b0a9fb7b641b045fe4d" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 } } peers { permanent_uuid: "0a104fc4fb1c421eb7352788001778df" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } } }
I20260812 11:14:32.856623 20463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a104fc4fb1c421eb7352788001778df [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:14:32.858127 20413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "53ed5eda1e2d4f128b71158260a48f90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ed5eda1e2d4f128b71158260a48f90" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44427 } } peers { permanent_uuid: "a0158d9002234b0a9fb7b641b045fe4d" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 } } peers { permanent_uuid: "0a104fc4fb1c421eb7352788001778df" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } } }
I20260812 11:14:32.858234 20413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:32.861222 20439 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0158d9002234b0a9fb7b641b045fe4d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "53ed5eda1e2d4f128b71158260a48f90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ed5eda1e2d4f128b71158260a48f90" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 44427 } } peers { permanent_uuid: "a0158d9002234b0a9fb7b641b045fe4d" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45173 } } peers { permanent_uuid: "0a104fc4fb1c421eb7352788001778df" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 45509 } } }
I20260812 11:14:32.861339 20439 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0158d9002234b0a9fb7b641b045fe4d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:14:32.862278 20506 mvcc.cc:205] Tried to move back new op lower bound from 7317640285485150208 to 7317640285363122176. Current Snapshot: MvccSnapshot[applied={T|T < 7317640285485150208}]
I20260812 11:14:32.862618 20484 catalog_manager.cc:1347] Generated new cluster ID: 8ca8b11732564f03a4201bb959b1d8f4
I20260812 11:14:32.862677 20484 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:14:32.896824 20484 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:14:32.898270 20484 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:14:32.917205 20484 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53ed5eda1e2d4f128b71158260a48f90: Generated new TSK 0
I20260812 11:14:32.918107 20484 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 11:14:32.960012 20157 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:32.956137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51966 (local address 127.17.44.254:44427)
0812 11:14:32.956304 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:32.956309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:32.958623 (+ 2314us) server_negotiation.cc:407] Connection header received
0812 11:14:32.958862 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:32.958866 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:32.958921 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:32.959035 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:32.959873 (+ 838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51966: BlockingRecv error: recv got EOF from 127.0.0.1:51966 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260812 11:14:32.961700 20277 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:32.956640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60338 (local address 127.17.44.253:45173)
0812 11:14:32.956767 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:32.956772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:32.960717 (+ 3945us) server_negotiation.cc:407] Connection header received
0812 11:14:32.960908 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:32.960912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:32.960990 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:32.961048 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:32.961583 (+ 535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60338: BlockingRecv error: recv got EOF from 127.0.0.1:60338 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260812 11:14:32.963255 20408 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:32.957826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60370 (local address 127.17.44.252:45509)
0812 11:14:32.957974 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:32.957979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:32.962343 (+ 4364us) server_negotiation.cc:407] Connection header received
0812 11:14:32.962514 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:32.962517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:32.962566 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:32.962619 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:32.963134 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60370: BlockingRecv error: recv got EOF from 127.0.0.1:60370 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260812 11:14:32.967923 20277 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:32.966355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60342 (local address 127.17.44.253:45173)
0812 11:14:32.966459 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:32.966463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:32.967007 (+ 544us) server_negotiation.cc:407] Connection header received
0812 11:14:32.967208 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:32.967211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:32.967296 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:32.967345 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:32.967832 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60342: BlockingRecv error: recv got EOF from 127.0.0.1:60342 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260812 11:14:32.977134 20277 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:32.975737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60354 (local address 127.17.44.253:45173)
0812 11:14:32.975865 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:32.975872 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:32.976221 (+ 349us) server_negotiation.cc:407] Connection header received
0812 11:14:32.976420 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:32.976423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:32.976473 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:32.976522 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:32.977028 (+ 506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60354: BlockingRecv error: recv got EOF from 127.0.0.1:60354 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260812 11:14:32.985702 20277 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:32.983625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60370 (local address 127.17.44.253:45173)
0812 11:14:32.983733 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:32.983737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:32.984440 (+ 703us) server_negotiation.cc:407] Connection header received
0812 11:14:32.984657 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:32.984660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:32.984726 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:32.984810 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:32.985594 (+ 784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60370: BlockingRecv error: recv got EOF from 127.0.0.1:60370 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260812 11:14:32.996958 20277 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:32.995342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60386 (local address 127.17.44.253:45173)
0812 11:14:32.995472 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:32.995476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:32.996034 (+ 558us) server_negotiation.cc:407] Connection header received
0812 11:14:32.996214 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:32.996217 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:32.996265 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:32.996314 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:32.996851 (+ 537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60386: BlockingRecv error: recv got EOF from 127.0.0.1:60386 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260812 11:14:33.008642 20277 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:33.007455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60396 (local address 127.17.44.253:45173)
0812 11:14:33.007570 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:33.007574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:33.007801 (+ 227us) server_negotiation.cc:407] Connection header received
0812 11:14:33.007962 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:33.007966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:33.008018 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:33.008107 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:33.008524 (+ 417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60396: BlockingRecv error: recv got EOF from 127.0.0.1:60396 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 11:14:33.028244 20277 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:33.026694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60402 (local address 127.17.44.253:45173)
0812 11:14:33.026839 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:33.026844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:33.027142 (+ 298us) server_negotiation.cc:407] Connection header received
0812 11:14:33.027314 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:33.027317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:33.027374 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:33.027464 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:33.028037 (+ 573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60402: BlockingRecv error: recv got EOF from 127.0.0.1:60402 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260812 11:14:33.064730 20277 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:33.063388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60404 (local address 127.17.44.253:45173)
0812 11:14:33.063567 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:33.063573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:33.063716 (+ 143us) server_negotiation.cc:407] Connection header received
0812 11:14:33.063907 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:33.063910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:33.063973 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:33.064075 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:33.064571 (+ 496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60404: BlockingRecv error: recv got EOF from 127.0.0.1:60404 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260812 11:14:33.132649 20277 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:33.131355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60414 (local address 127.17.44.253:45173)
0812 11:14:33.131586 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:33.131590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:33.131608 (+ 18us) server_negotiation.cc:407] Connection header received
0812 11:14:33.131821 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:33.131824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:33.131877 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:33.131968 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:33.132510 (+ 542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60414: BlockingRecv error: recv got EOF from 127.0.0.1:60414 (error 108)
Metrics: {"server-negotiator.queue_time_us":135}
W20260812 11:14:33.264165 20277 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:33.262841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60426 (local address 127.17.44.253:45173)
0812 11:14:33.263016 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:33.263020 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:33.263169 (+ 149us) server_negotiation.cc:407] Connection header received
0812 11:14:33.263293 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:33.263295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:33.263351 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:33.263435 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:33.263932 (+ 497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60426: BlockingRecv error: recv got EOF from 127.0.0.1:60426 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260812 11:14:33.523604 20277 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:33.522439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43638 (local address 127.17.44.253:45173)
0812 11:14:33.522600 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:33.522604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:33.522621 (+ 17us) server_negotiation.cc:407] Connection header received
0812 11:14:33.522788 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:33.522793 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:33.522847 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:33.522966 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:33.523450 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43638: BlockingRecv error: recv got EOF from 127.0.0.1:43638 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 11:14:33.785790 20458 catalog_manager.cc:1259] Loaded cluster ID: 8ca8b11732564f03a4201bb959b1d8f4
I20260812 11:14:33.785893 20458 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a0158d9002234b0a9fb7b641b045fe4d: loading cluster ID for follower catalog manager: success
I20260812 11:14:33.787664 20458 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a0158d9002234b0a9fb7b641b045fe4d: acquiring CA information for follower catalog manager: success
I20260812 11:14:33.788964 20458 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a0158d9002234b0a9fb7b641b045fe4d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 11:14:33.799207 20480 catalog_manager.cc:1259] Loaded cluster ID: 8ca8b11732564f03a4201bb959b1d8f4
I20260812 11:14:33.799361 20480 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0a104fc4fb1c421eb7352788001778df: loading cluster ID for follower catalog manager: success
I20260812 11:14:33.800921 20480 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0a104fc4fb1c421eb7352788001778df: acquiring CA information for follower catalog manager: success
I20260812 11:14:33.802034 20480 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0a104fc4fb1c421eb7352788001778df: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 11:14:34.043898 20627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:34.038219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43640 (local address 127.17.44.253:45173)
0812 11:14:34.038559 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:34.038563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:34.038581 (+ 18us) server_negotiation.cc:407] Connection header received
0812 11:14:34.039146 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:34.039151 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:34.039372 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:34.039545 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:14:34.040035 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:34.040605 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:34.041639 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:34.041921 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:34.042054 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:34.042518 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:14:34.043275 (+ 757us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:14:34.043400 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:34.043579 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 11:14:34.043711 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":104,"threads_started":1}
I20260812 11:14:34.052464 20637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:34.048165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42326 (local address 127.17.44.252:45509)
0812 11:14:34.048412 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:34.048416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:34.048431 (+ 15us) server_negotiation.cc:407] Connection header received
0812 11:14:34.048519 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:34.048522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:34.048663 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:34.048781 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:14:34.049115 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:34.049638 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:34.050551 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:34.050790 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:34.050883 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:34.051332 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:14:34.051819 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:14:34.051925 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:34.052199 (+ 274us) server_negotiation.cc:299] Negotiation successful
0812 11:14:34.052297 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":93,"threads_started":1}
I20260812 11:14:34.061821 20627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:34.056875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43648 (local address 127.17.44.253:45173)
0812 11:14:34.057089 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:34.057097 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:34.057364 (+ 267us) server_negotiation.cc:407] Connection header received
0812 11:14:34.057716 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:34.057724 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:34.057809 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:34.057917 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:14:34.058480 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:34.059355 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:34.060626 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:34.060905 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:34.061018 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:34.061297 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:14:34.061392 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:14:34.061476 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:34.061617 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 11:14:34.061682 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 11:14:34.062597 20637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:34.057990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42332 (local address 127.17.44.252:45509)
0812 11:14:34.058152 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:34.058157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:34.058362 (+ 205us) server_negotiation.cc:407] Connection header received
0812 11:14:34.058491 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:34.058497 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:34.058605 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:34.058688 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:14:34.059068 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:34.059973 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:34.061187 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:34.061506 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:34.061590 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:34.062128 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:14:34.062229 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:14:34.062306 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:34.062405 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 11:14:34.062463 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 11:14:34.062846 20644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:34.056725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57448 (local address 127.17.44.254:44427)
0812 11:14:34.057292 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:34.057298 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:34.057544 (+ 246us) server_negotiation.cc:407] Connection header received
0812 11:14:34.057640 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:34.057642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:34.057810 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:34.057985 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:14:34.058478 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:34.059357 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:34.060875 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:34.061186 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:34.061297 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:34.061738 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:14:34.062091 (+ 353us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:14:34.062182 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:34.062672 (+ 490us) server_negotiation.cc:299] Negotiation successful
0812 11:14:34.062739 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":110,"threads_started":1}
I20260812 11:14:34.066993 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 20044
I20260812 11:14:34.083738 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 20162
I20260812 11:14:34.102655 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 20280
2026-08-12T11:14:34Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3483 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 11:14:34 dist-test-slave-drl0 krb5kdc[20653](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 11:14:34 dist-test-slave-drl0 krb5kdc[20653](info): set up 2 sockets
Aug 12 11:14:34 dist-test-slave-drl0 krb5kdc[20653](info): commencing operation
krb5kdc: starting...
W20260812 11:14:35.425627 17587 mini_kdc.cc:121] Time spent starting KDC: real 1.294s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/krb5kdc/test-user.keytab.
Aug 12 11:14:35 dist-test-slave-drl0 krb5kdc[20653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T11:14:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T11:14:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
Entry for principal kudu/127.17.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab.
I20260812 11:14:35.558084 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:33495
--webserver_interface=127.17.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:33495,127.17.44.253:45925,127.17.44.252:37947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 11:14:35.673707 20676 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 11:14:35.673978 20676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:14:35.674002 20676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:14:35.674038 20676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:14:35.678072 20676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:14:35.678143 20676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:14:35.678164 20676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:14:35.678181 20676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:14:35.678196 20676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:14:35.683691 20676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:33495,127.17.44.253:45925,127.17.44.252:37947
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.254.keytab
--principal=kudu/127.17.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.254:33495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:14:35.684691 20676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:14:35.685696 20676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:14:35.694419 20684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:35.694427 20685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:35.694536 20676 server_base.cc:1034] running on GCE node
W20260812 11:14:35.694481 20689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:35.695483 20676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:14:35.696516 20676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:14:35.697786 20676 hybrid_clock.cc:648] HybridClock initialized: now 1786533275697734 us; error 71 us; skew 500 ppm
Aug 12 11:14:35 dist-test-slave-drl0 krb5kdc[20653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533275, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:14:35.701182 20676 init.cc:377] Logged in from keytab as kudu/127.17.44.254@KRBTEST.COM (short username kudu)
I20260812 11:14:35.702097 20676 webserver.cc:460] Webserver started at http://127.17.44.254:34235/ using document root <none> and password file <none>
I20260812 11:14:35.702625 20676 fs_manager.cc:359] Metadata directory not provided
I20260812 11:14:35.702684 20676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:14:35.702841 20676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:14:35.704890 20676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-0/data/instance:
uuid: "af6c1021bab048c280b93b4d48c95b46"
format_stamp: "Formatted at 2026-08-12 11:14:35 on dist-test-slave-drl0"
server_key: "1244b34fb748254e3e1a2151bad322de"
server_key_iv: "c63abede53c3c3b22ceccc134611509c"
server_key_version: "encryptionkey@0"
I20260812 11:14:35.705401 20676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-0/wal/instance:
uuid: "af6c1021bab048c280b93b4d48c95b46"
format_stamp: "Formatted at 2026-08-12 11:14:35 on dist-test-slave-drl0"
server_key: "1244b34fb748254e3e1a2151bad322de"
server_key_iv: "c63abede53c3c3b22ceccc134611509c"
server_key_version: "encryptionkey@0"
I20260812 11:14:35.709263 20676 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 11:14:35.711714 20698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:14:35.712949 20676 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 11:14:35.713090 20676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-0/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-0/wal
uuid: "af6c1021bab048c280b93b4d48c95b46"
format_stamp: "Formatted at 2026-08-12 11:14:35 on dist-test-slave-drl0"
server_key: "1244b34fb748254e3e1a2151bad322de"
server_key_iv: "c63abede53c3c3b22ceccc134611509c"
server_key_version: "encryptionkey@0"
I20260812 11:14:35.713202 20676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:14:35.743217 20676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:14:35.747081 20676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:14:35.747370 20676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:14:35.755206 20676 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.254:33495
I20260812 11:14:35.755513 20789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.254:33495 every 8 connection(s)
I20260812 11:14:35.756213 20676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-0/data/info.pb
I20260812 11:14:35.759081 20791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:14:35.762802 20791 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33495 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:14:35.766211 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 20676
I20260812 11:14:35.766352 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-0/wal/instance
I20260812 11:14:35.766647 17587 external_mini_cluster.cc:1428] Setting key 386e99659d620f6414300b7b90f908f4
Aug 12 11:14:35 dist-test-slave-drl0 krb5kdc[20653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533275, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.44.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.253@KRBTEST.COM" created.
I20260812 11:14:35.781065 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:35.765114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33495 (local address 127.0.0.1:43830)
0812 11:14:35.766002 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:35.766050 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 11:14:35.767242 (+ 1192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:14:35.768043 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:14:35.768048 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:14:35.768518 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:14:35.768791 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:35.768813 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:35.770006 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:35.770013 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:35.770990 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:35.771006 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:35.772234 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:35.772241 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:35.772411 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:35.773210 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:14:35.773236 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:14:35.774924 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:14:35.777416 (+ 2492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:35.777425 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:35.777432 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:35.777661 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:35.778422 (+ 761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:35.778425 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:35.778427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:35.778551 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:35.779569 (+ 1018us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:14:35.779577 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:14:35.779712 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 11:14:35.779895 (+ 183us) client_negotiation.cc:240] Negotiation successful
0812 11:14:35.780224 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"spinlock_wait_cycles":25984,"thread_start_us":279,"threads_started":1}
I20260812 11:14:35.781062 20793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:35.765114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43830 (local address 127.17.44.254:33495)
0812 11:14:35.765676 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:35.765696 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:35.766475 (+ 779us) server_negotiation.cc:407] Connection header received
0812 11:14:35.767656 (+ 1181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:35.767661 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:35.767854 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:35.768037 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:35.769000 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:35.769867 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:35.771135 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:35.771837 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:35.772033 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:35.775154 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:35.775165 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:35.775178 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:35.775210 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:35.777291 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:35.778017 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:35.778028 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:35.778029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:35.778272 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:35.778703 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:35.778706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:35.778715 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:35.779310 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:35.779618 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:35.780023 (+ 405us) server_negotiation.cc:299] Negotiation successful
0812 11:14:35.780321 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":113,"server-negotiator.queue_time_us":377,"thread_start_us":137,"threads_started":1}
I20260812 11:14:35.783856 20791 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:14:35.785043 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:35.784650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:45925 (local address 127.0.0.1:46144)
0812 11:14:35.784848 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:35.784946 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.253:45925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260812 11:14:35.785313 20700 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.44.253:45925: connect: Connection refused (error 111)
I20260812 11:14:35.786408 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:35.786134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:45925 (local address 127.0.0.1:46154)
0812 11:14:35.786281 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:35.786336 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.253:45925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 11:14:35.786590 20791 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.253:45925: Network error: Client connection negotiation failed: client connection to 127.17.44.253:45925: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.253.keytab.
Entry for principal kudu/127.17.44.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.253.keytab.
I20260812 11:14:35.798507 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.253:45925
--webserver_interface=127.17.44.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.253.keytab
--principal=kudu/127.17.44.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:33495,127.17.44.253:45925,127.17.44.252:37947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:14:35.835878 20791 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } attempt: 1
I20260812 11:14:35.836820 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:35.836461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:45925 (local address 127.0.0.1:46170)
0812 11:14:35.836645 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:35.836727 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.253:45925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260812 11:14:35.837621 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:35.837368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:45925 (local address 127.0.0.1:46186)
0812 11:14:35.837501 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:35.837553 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.253:45925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260812 11:14:35.837910 20791 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.253:45925: Network error: Client connection negotiation failed: client connection to 127.17.44.253:45925: connect: Connection refused (error 111)
I20260812 11:14:35.906211 20791 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } attempt: 2
I20260812 11:14:35.907135 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:35.906755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:45925 (local address 127.0.0.1:46200)
0812 11:14:35.906969 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:35.907041 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.253:45925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260812 11:14:35.907959 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:35.907712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:45925 (local address 127.0.0.1:46216)
0812 11:14:35.907834 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:35.907891 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.253:45925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 11:14:35.908123 20791 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.253:45925: Network error: Client connection negotiation failed: client connection to 127.17.44.253:45925: connect: Connection refused (error 111)
W20260812 11:14:35.939157 20800 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 11:14:35.939429 20800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:14:35.939455 20800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:14:35.939493 20800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 11:14:35.943569 20800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:14:35.943629 20800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:14:35.943650 20800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:14:35.943666 20800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:14:35.943681 20800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:14:35.949924 20800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:33495,127.17.44.253:45925,127.17.44.252:37947
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.253.keytab
--principal=kudu/127.17.44.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.253:45925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.44.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:14:35.951073 20800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:14:35.952108 20800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:14:35.962013 20812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:35.962162 20815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:35.962356 20808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:35.962338 20800 server_base.cc:1034] running on GCE node
I20260812 11:14:35.963243 20800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:14:35.964299 20800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:14:35.965507 20800 hybrid_clock.cc:648] HybridClock initialized: now 1786533275965470 us; error 61 us; skew 500 ppm
Aug 12 11:14:35 dist-test-slave-drl0 krb5kdc[20653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533275, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:14:35.968712 20800 init.cc:377] Logged in from keytab as kudu/127.17.44.253@KRBTEST.COM (short username kudu)
I20260812 11:14:35.970124 20800 webserver.cc:460] Webserver started at http://127.17.44.253:37547/ using document root <none> and password file <none>
I20260812 11:14:35.970747 20800 fs_manager.cc:359] Metadata directory not provided
I20260812 11:14:35.970817 20800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:14:35.971076 20800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:14:35.973157 20800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-1/data/instance:
uuid: "820895ecbc5941f2bf5349707ca681b6"
format_stamp: "Formatted at 2026-08-12 11:14:35 on dist-test-slave-drl0"
server_key: "bf31018be6cd96fe85090d0f6fd7295d"
server_key_iv: "4c1f9cc1be8bd4c63a2553055dec9f1b"
server_key_version: "encryptionkey@0"
I20260812 11:14:35.973734 20800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-1/wal/instance:
uuid: "820895ecbc5941f2bf5349707ca681b6"
format_stamp: "Formatted at 2026-08-12 11:14:35 on dist-test-slave-drl0"
server_key: "bf31018be6cd96fe85090d0f6fd7295d"
server_key_iv: "4c1f9cc1be8bd4c63a2553055dec9f1b"
server_key_version: "encryptionkey@0"
I20260812 11:14:35.978152 20800 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 11:14:35.980803 20832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:14:35.982115 20800 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 11:14:35.982283 20800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-1/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-1/wal
uuid: "820895ecbc5941f2bf5349707ca681b6"
format_stamp: "Formatted at 2026-08-12 11:14:35 on dist-test-slave-drl0"
server_key: "bf31018be6cd96fe85090d0f6fd7295d"
server_key_iv: "4c1f9cc1be8bd4c63a2553055dec9f1b"
server_key_version: "encryptionkey@0"
I20260812 11:14:35.982398 20800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:14:35.999342 20791 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } attempt: 3
I20260812 11:14:36.000085 20800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:14:36.000167 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:35.999843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:45925 (local address 127.0.0.1:46224)
0812 11:14:36.000005 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:36.000073 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.253:45925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 11:14:36.001247 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.000973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:45925 (local address 127.0.0.1:46230)
0812 11:14:36.001127 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:36.001183 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.253:45925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260812 11:14:36.001437 20791 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.253:45925: Network error: Client connection negotiation failed: client connection to 127.17.44.253:45925: connect: Connection refused (error 111)
I20260812 11:14:36.003613 20800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:14:36.003805 20800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:14:36.011536 20800 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.253:45925
I20260812 11:14:36.011930 20911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.253:45925 every 8 connection(s)
I20260812 11:14:36.012636 20800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-1/data/info.pb
I20260812 11:14:36.015627 20915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:14:36.019210 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 20800
I20260812 11:14:36.019315 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-1/wal/instance
I20260812 11:14:36.019613 17587 external_mini_cluster.cc:1428] Setting key 951b2ba1cce7bcd4af23272545fd0377
I20260812 11:14:36.020499 20915 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33495 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.44.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.44.252@KRBTEST.COM" created.
Aug 12 11:14:36 dist-test-slave-drl0 krb5kdc[20653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533275, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.253@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:14:36.040194 20793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.022767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43832 (local address 127.17.44.254:33495)
0812 11:14:36.022952 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.022957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.023867 (+ 910us) server_negotiation.cc:407] Connection header received
0812 11:14:36.025410 (+ 1543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.025412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.025464 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.025555 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.027541 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.028087 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.029869 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.030116 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.030207 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:36.034545 (+ 4338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:36.034554 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:36.034562 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:36.034603 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:36.037362 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.038242 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.038246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.038248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.038346 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.038834 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.038837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.038838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.039114 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:36.039203 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:36.039949 (+ 746us) server_negotiation.cc:299] Negotiation successful
0812 11:14:36.040070 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 11:14:36.041469 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.023174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33495 (local address 127.0.0.1:43832)
0812 11:14:36.023670 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:36.023728 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 11:14:36.025016 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:14:36.026008 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:14:36.026020 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:14:36.026575 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:14:36.027346 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:36.027368 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.028239 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.028244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:36.029698 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:36.029714 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.030296 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.030300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:36.030449 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:36.031120 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:14:36.031137 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:14:36.034196 (+ 3059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:14:36.037539 (+ 3343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:36.037549 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:36.037570 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:36.038048 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:36.038479 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:36.038483 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:36.038486 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:36.038684 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:36.039232 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:14:36.039242 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:14:36.039615 (+ 373us) client_negotiation.cc:769] Sending connection context
0812 11:14:36.039861 (+ 246us) client_negotiation.cc:240] Negotiation successful
0812 11:14:36.040248 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":101,"threads_started":1}
I20260812 11:14:36.043550 20915 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 11:14:36 dist-test-slave-drl0 krb5kdc[20653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533275, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.253@KRBTEST.COM for kudu/127.17.44.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.44.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.252.keytab.
Entry for principal kudu/127.17.44.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.252.keytab.
I20260812 11:14:36.052513 17587 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
/tmp/dist-test-taskqOACJ0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.252:37947
--webserver_interface=127.17.44.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.252.keytab
--principal=kudu/127.17.44.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.44.212:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.44.254:33495,127.17.44.253:45925,127.17.44.252:37947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 11:14:36.054690 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.044401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:45925 (local address 127.0.0.1:46238)
0812 11:14:36.044574 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:36.044589 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 11:14:36.044661 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:14:36.045647 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:14:36.045650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:14:36.045793 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:14:36.045908 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:36.045913 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.047173 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.047176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:36.047755 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:36.047764 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.048521 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.048525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:36.048603 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:36.049130 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:14:36.049142 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:14:36.050590 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:14:36.052997 (+ 2407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:36.053003 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:36.053007 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:36.053194 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:36.053499 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:36.053502 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:36.053504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:36.053581 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:36.054048 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:14:36.054051 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:14:36.054237 (+ 186us) client_negotiation.cc:769] Sending connection context
0812 11:14:36.054363 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 11:14:36.054508 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260812 11:14:36.054746 20922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.044597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46238 (local address 127.17.44.253:45925)
0812 11:14:36.045111 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.045129 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.045156 (+ 27us) server_negotiation.cc:407] Connection header received
0812 11:14:36.045206 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.045212 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.045421 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.045648 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.046063 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.047072 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.047882 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.048376 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.048472 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:36.050740 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:36.050748 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:36.050760 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:36.050781 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:36.052885 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.053309 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.053315 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.053316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.053397 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.053678 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.053681 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.053683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.053926 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:36.054131 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:36.054458 (+ 327us) server_negotiation.cc:299] Negotiation successful
0812 11:14:36.054603 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":290,"threads_started":1}
I20260812 11:14:36.056890 20915 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:14:36.057633 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.057368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:37947 (local address 127.0.0.1:45708)
0812 11:14:36.057497 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:36.057576 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:37947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 11:14:36.057876 20836 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.44.252:37947: connect: Connection refused (error 111)
I20260812 11:14:36.059115 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.058810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:37947 (local address 127.0.0.1:45718)
0812 11:14:36.059006 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:36.059062 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:37947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260812 11:14:36.059427 20915 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.252:37947: Network error: Client connection negotiation failed: client connection to 127.17.44.252:37947: connect: Connection refused (error 111)
I20260812 11:14:36.108822 20915 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } attempt: 1
I20260812 11:14:36.109756 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.109429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:37947 (local address 127.0.0.1:45728)
0812 11:14:36.109584 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:36.109650 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:37947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 11:14:36.110827 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.110378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:37947 (local address 127.0.0.1:45744)
0812 11:14:36.110631 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:36.110693 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:37947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
W20260812 11:14:36.111145 20915 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.252:37947: Network error: Client connection negotiation failed: client connection to 127.17.44.252:37947: connect: Connection refused (error 111)
I20260812 11:14:36.144661 20791 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } attempt: 4
Aug 12 11:14:36 dist-test-slave-drl0 krb5kdc[20653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533275, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for kudu/127.17.44.253@KRBTEST.COM
I20260812 11:14:36.155453 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.145302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:45925 (local address 127.0.0.1:46250)
0812 11:14:36.145561 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:36.145577 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:14:36.145674 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:14:36.146110 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:14:36.146115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:14:36.146464 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:14:36.146748 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:36.146757 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.147754 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.147759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:36.148619 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:36.148630 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.149143 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.149147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:36.149253 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:36.149786 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:14:36.149805 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:14:36.151639 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:14:36.153773 (+ 2134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:36.153778 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:36.153780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:36.154013 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:36.154329 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:36.154333 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:36.154335 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:36.154421 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:36.154827 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:14:36.154832 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:14:36.154993 (+ 161us) client_negotiation.cc:769] Sending connection context
0812 11:14:36.155121 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 11:14:36.155275 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172}
I20260812 11:14:36.155453 20922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.145525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46250 (local address 127.17.44.253:45925)
0812 11:14:36.145665 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.145669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.145685 (+ 16us) server_negotiation.cc:407] Connection header received
0812 11:14:36.145776 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.145778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.145820 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.145908 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.146911 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.147632 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.148776 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.149019 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.149109 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:36.151888 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:36.151892 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:36.151894 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:36.151927 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:36.153624 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.154128 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.154132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.154134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.154222 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.154521 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.154524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.154526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.154713 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:36.154822 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:36.155177 (+ 355us) server_negotiation.cc:299] Negotiation successful
0812 11:14:36.155296 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 11:14:36.157342 20791 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:14:36.158099 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.157850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:37947 (local address 127.0.0.1:45746)
0812 11:14:36.157976 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:36.158037 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:37947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 11:14:36.158847 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.158595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:37947 (local address 127.0.0.1:45752)
0812 11:14:36.158732 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:36.158796 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:37947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260812 11:14:36.159050 20791 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.252:37947: Network error: Client connection negotiation failed: client connection to 127.17.44.252:37947: connect: Connection refused (error 111)
W20260812 11:14:36.177210 20924 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 11:14:36.177520 20924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 11:14:36.177546 20924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 11:14:36.177593 20924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 11:14:36.179368 20915 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } attempt: 2
I20260812 11:14:36.180379 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.180102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:37947 (local address 127.0.0.1:45758)
0812 11:14:36.180240 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:36.180308 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:37947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 11:14:36.181267 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.181029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:37947 (local address 127.0.0.1:45772)
0812 11:14:36.181157 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:36.181212 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:37947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260812 11:14:36.181452 20915 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.252:37947: Network error: Client connection negotiation failed: client connection to 127.17.44.252:37947: connect: Connection refused (error 111)
W20260812 11:14:36.183169 20924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 11:14:36.183223 20924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 11:14:36.183243 20924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 11:14:36.183259 20924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 11:14:36.183324 20924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 11:14:36.189873 20924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.44.212:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.44.254:33495,127.17.44.253:45925,127.17.44.252:37947
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/krb5kdc/kudu_127.17.44.252.keytab
--principal=kudu/127.17.44.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.44.252:37947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.44.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 10:26:14 UTC on ccycloud.quasar-hqgckh.root.comops.site
build id 4582280
I20260812 11:14:36.191350 20924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 11:14:36.192591 20924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 11:14:36.201560 20933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:36.201553 20936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 11:14:36.201586 20932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 11:14:36.202234 20924 server_base.cc:1034] running on GCE node
I20260812 11:14:36.203050 20924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 11:14:36.204109 20924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 11:14:36.205345 20924 hybrid_clock.cc:648] HybridClock initialized: now 1786533276205326 us; error 36 us; skew 500 ppm
Aug 12 11:14:36 dist-test-slave-drl0 krb5kdc[20653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533276, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 11:14:36.208592 20924 init.cc:377] Logged in from keytab as kudu/127.17.44.252@KRBTEST.COM (short username kudu)
I20260812 11:14:36.210080 20924 webserver.cc:460] Webserver started at http://127.17.44.252:40559/ using document root <none> and password file <none>
I20260812 11:14:36.211346 20924 fs_manager.cc:359] Metadata directory not provided
I20260812 11:14:36.211457 20924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 11:14:36.211683 20924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 11:14:36.214205 20924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-2/data/instance:
uuid: "339d6aed629d406f9d4d4837fdcc2a02"
format_stamp: "Formatted at 2026-08-12 11:14:36 on dist-test-slave-drl0"
server_key: "bd09ff088089a91530fb502dd54ea3d6"
server_key_iv: "3c1f5acec5c6ebfb1307432ae14c631a"
server_key_version: "encryptionkey@0"
I20260812 11:14:36.215127 20924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-2/wal/instance:
uuid: "339d6aed629d406f9d4d4837fdcc2a02"
format_stamp: "Formatted at 2026-08-12 11:14:36 on dist-test-slave-drl0"
server_key: "bd09ff088089a91530fb502dd54ea3d6"
server_key_iv: "3c1f5acec5c6ebfb1307432ae14c631a"
server_key_version: "encryptionkey@0"
I20260812 11:14:36.218333 20791 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } attempt: 1
I20260812 11:14:36.219209 20924 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 11:14:36.219416 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.219049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:37947 (local address 127.0.0.1:45776)
0812 11:14:36.219223 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:36.219297 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:37947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 11:14:36.220515 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.220173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:37947 (local address 127.0.0.1:45788)
0812 11:14:36.220332 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:36.220407 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.44.252:37947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260812 11:14:36.220748 20791 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.44.252:37947: Network error: Client connection negotiation failed: client connection to 127.17.44.252:37947: connect: Connection refused (error 111)
I20260812 11:14:36.222011 20954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 11:14:36.223264 20924 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 11:14:36.223409 20924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-2/data,/tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-2/wal
uuid: "339d6aed629d406f9d4d4837fdcc2a02"
format_stamp: "Formatted at 2026-08-12 11:14:36 on dist-test-slave-drl0"
server_key: "bd09ff088089a91530fb502dd54ea3d6"
server_key_iv: "3c1f5acec5c6ebfb1307432ae14c631a"
server_key_version: "encryptionkey@0"
I20260812 11:14:36.223520 20924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 11:14:36.244150 20924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 11:14:36.248145 20924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 11:14:36.248543 20924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 11:14:36.256647 20924 rpc_server.cc:307] RPC server started. Bound to: 127.17.44.252:37947
I20260812 11:14:36.256944 21032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.44.252:37947 every 8 connection(s)
I20260812 11:14:36.257736 20924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-2/data/info.pb
I20260812 11:14:36.260532 21033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 11:14:36.262476 17587 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu as pid 20924
I20260812 11:14:36.262640 17587 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqOACJ0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786532961723938-17587-0/minicluster-data/master-2/wal/instance
I20260812 11:14:36.262988 17587 external_mini_cluster.cc:1428] Setting key 9723d522aaa3833f1ad17a07ff6489fc
I20260812 11:14:36.265007 21033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33495 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:14:36.272779 20915 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } attempt: 3
Aug 12 11:14:36 dist-test-slave-drl0 krb5kdc[20653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.252@KRBTEST.COM
Aug 12 11:14:36 dist-test-slave-drl0 krb5kdc[20653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533276, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.252@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
Aug 12 11:14:36 dist-test-slave-drl0 krb5kdc[20653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533275, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.253@KRBTEST.COM for kudu/127.17.44.252@KRBTEST.COM
I20260812 11:14:36.279088 20793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.267778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43840 (local address 127.17.44.254:33495)
0812 11:14:36.267936 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.267940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.268218 (+ 278us) server_negotiation.cc:407] Connection header received
0812 11:14:36.268344 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.268347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.268398 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.268496 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.269290 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.269761 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.271063 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.271440 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.271558 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:36.275147 (+ 3589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:36.275150 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:36.275153 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:36.275180 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:36.276960 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.277549 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.277555 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.277560 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.277656 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.278006 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.278011 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.278014 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.278221 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:36.278340 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:36.278763 (+ 423us) server_negotiation.cc:299] Negotiation successful
0812 11:14:36.278881 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 11:14:36.280216 21039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.264384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45790 (local address 127.17.44.252:37947)
0812 11:14:36.264852 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.264873 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.264916 (+ 43us) server_negotiation.cc:407] Connection header received
0812 11:14:36.265823 (+ 907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.265849 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.266262 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.266605 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.267566 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.268889 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.269992 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.270587 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.270908 (+ 321us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:36.274477 (+ 3569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:36.274484 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:36.274496 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:36.274527 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:36.277079 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.277607 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.277616 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.277619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.277749 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.278097 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.278099 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.278101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.278584 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:36.279732 (+ 1148us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:36.279836 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 11:14:36.280007 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":280,"spinlock_wait_cycles":3456,"thread_start_us":115,"threads_started":1}
I20260812 11:14:36.280328 21040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.267644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.254:33495 (local address 127.0.0.1:43840)
0812 11:14:36.268082 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:36.268130 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 11:14:36.268249 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:14:36.268548 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:14:36.268552 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:14:36.268855 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:14:36.269020 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:36.269029 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.269919 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.269924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:36.270902 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:36.270912 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.271861 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.271866 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:36.272029 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:36.272673 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:14:36.272700 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:14:36.274909 (+ 2209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:14:36.277092 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:36.277101 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:36.277113 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:36.277446 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:36.277783 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:36.277785 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:36.277786 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:36.277897 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:36.278339 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:14:36.278346 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:14:36.278544 (+ 198us) client_negotiation.cc:769] Sending connection context
0812 11:14:36.278718 (+ 174us) client_negotiation.cc:240] Negotiation successful
0812 11:14:36.279045 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":28,"thread_start_us":129,"threads_started":1}
I20260812 11:14:36.281764 21033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } has no permanent_uuid. Determining permanent_uuid...
I20260812 11:14:36.283454 21041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.273593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45794 (local address 127.17.44.252:37947)
0812 11:14:36.273920 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.273939 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.273952 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:14:36.273986 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.273989 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.274034 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.274148 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.274629 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.275426 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.276517 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.276837 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.276976 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:36.279463 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:36.279466 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:36.279468 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:36.279491 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:36.281212 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.282099 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.282106 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.282110 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.282199 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.282553 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.282556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.282557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.282716 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:36.282877 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:36.283246 (+ 369us) server_negotiation.cc:299] Negotiation successful
0812 11:14:36.283340 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":98,"threads_started":1}
I20260812 11:14:36.283469 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.273531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:37947 (local address 127.0.0.1:45794)
0812 11:14:36.273643 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:36.273670 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 11:14:36.273798 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:14:36.274197 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:14:36.274201 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:14:36.274380 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:14:36.274505 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:36.274514 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.275562 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.275569 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:36.276373 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:36.276383 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.276978 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.276982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:36.277068 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:36.277807 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:14:36.277826 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:14:36.279295 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:14:36.281312 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:36.281315 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:36.281318 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:36.281568 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:36.282331 (+ 763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:36.282335 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:36.282338 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:36.282437 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:36.282870 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:14:36.282873 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:14:36.283031 (+ 158us) client_negotiation.cc:769] Sending connection context
0812 11:14:36.283143 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 11:14:36.283287 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":3}
I20260812 11:14:36.286839 20915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 820895ecbc5941f2bf5349707ca681b6: Bootstrap starting.
I20260812 11:14:36.287920 20791 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } attempt: 2
Aug 12 11:14:36 dist-test-slave-drl0 krb5kdc[20653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533276, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.252@KRBTEST.COM for kudu/127.17.44.253@KRBTEST.COM
Aug 12 11:14:36 dist-test-slave-drl0 krb5kdc[20653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.254@KRBTEST.COM
I20260812 11:14:36.291086 20915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 820895ecbc5941f2bf5349707ca681b6: Neither blocks nor log segments found. Creating new log.
I20260812 11:14:36.292066 20915 log.cc:824] T 00000000000000000000000000000000 P 820895ecbc5941f2bf5349707ca681b6: Log is configured to *not* fsync() on all Append() calls
I20260812 11:14:36.292852 20922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.282333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46252 (local address 127.17.44.253:45925)
0812 11:14:36.282632 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.282636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.282651 (+ 15us) server_negotiation.cc:407] Connection header received
0812 11:14:36.282683 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.282686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.282734 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.282863 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.284102 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.284885 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.286235 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.286571 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.286686 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:36.288966 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:36.288969 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:36.288972 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:36.288996 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:36.290827 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.291416 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.291420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.291421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.291506 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.291822 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.291825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.291827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.292079 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:36.292171 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:36.292544 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 11:14:36.292663 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":225}
I20260812 11:14:36.292985 20793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.283057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43852 (local address 127.17.44.254:33495)
0812 11:14:36.283157 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.283161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.283328 (+ 167us) server_negotiation.cc:407] Connection header received
0812 11:14:36.283367 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.283370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.283409 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.283475 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.284292 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.285007 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.286253 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.286470 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.286591 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:36.289462 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:36.289465 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:36.289468 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:36.289498 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:36.291069 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.291727 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.291730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.291732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.291815 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.292109 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.292112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.292114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.292320 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:36.292530 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:36.292671 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 11:14:36.292788 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 11:14:36.293465 21040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.282230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.253:45925 (local address 127.0.0.1:46252)
0812 11:14:36.282393 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:36.282407 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 11:14:36.282472 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:14:36.283631 (+ 1159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:14:36.283637 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:14:36.283817 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:14:36.283955 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:36.283964 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.285022 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.285026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:36.286105 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:36.286116 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.286843 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.286848 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:36.287000 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:36.287411 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:14:36.287427 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:14:36.288804 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:14:36.291011 (+ 2207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:36.291016 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:36.291019 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:36.291282 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:36.291621 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:36.291624 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:36.291626 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:36.291715 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:36.292237 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:14:36.292239 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:14:36.292342 (+ 103us) client_negotiation.cc:769] Sending connection context
0812 11:14:36.293160 (+ 818us) client_negotiation.cc:240] Negotiation successful
0812 11:14:36.293318 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260812 11:14:36.294113 21033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 11:14:36 dist-test-slave-drl0 krb5kdc[20653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533275, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.254@KRBTEST.COM for kudu/127.17.44.252@KRBTEST.COM
I20260812 11:14:36.297572 20915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 820895ecbc5941f2bf5349707ca681b6: No bootstrap required, opened a new log
Aug 12 11:14:36 dist-test-slave-drl0 krb5kdc[20653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533276, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.44.252@KRBTEST.COM for kudu/127.17.44.252@KRBTEST.COM
I20260812 11:14:36.301951 20915 raft_consensus.cc:348] T 00000000000000000000000000000000 P 820895ecbc5941f2bf5349707ca681b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6c1021bab048c280b93b4d48c95b46" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33495 } } peers { permanent_uuid: "820895ecbc5941f2bf5349707ca681b6" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } } peers { permanent_uuid: "339d6aed629d406f9d4d4837fdcc2a02" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } }
I20260812 11:14:36.302227 20915 raft_consensus.cc:729] T 00000000000000000000000000000000 P 820895ecbc5941f2bf5349707ca681b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 820895ecbc5941f2bf5349707ca681b6, State: Initialized, Role: FOLLOWER
I20260812 11:14:36.302472 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.288283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:37947 (local address 127.0.0.1:45800)
0812 11:14:36.288427 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:36.288443 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 11:14:36.288528 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:14:36.292817 (+ 4289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:14:36.292820 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:14:36.293000 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:14:36.293115 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:36.293122 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.294111 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.294115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:36.294888 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:36.294899 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.295563 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.295567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:36.295647 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:36.296211 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:14:36.296227 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:14:36.297926 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:14:36.300650 (+ 2724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:36.300655 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:36.300660 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:36.300955 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:36.301318 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:36.301323 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:36.301326 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:36.301435 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:36.301918 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:14:36.301923 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:14:36.302034 (+ 111us) client_negotiation.cc:769] Sending connection context
0812 11:14:36.302128 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 11:14:36.302265 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260812 11:14:36.302604 20915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 820895ecbc5941f2bf5349707ca681b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6c1021bab048c280b93b4d48c95b46" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33495 } } peers { permanent_uuid: "820895ecbc5941f2bf5349707ca681b6" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } } peers { permanent_uuid: "339d6aed629d406f9d4d4837fdcc2a02" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } }
I20260812 11:14:36.303097 21041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.288485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45800 (local address 127.17.44.252:37947)
0812 11:14:36.292580 (+ 4095us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.292584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.292595 (+ 11us) server_negotiation.cc:407] Connection header received
0812 11:14:36.292630 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.292635 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.292674 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.293225 (+ 551us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.293314 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.293989 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.295072 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.295465 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.295564 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:36.298084 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:36.298088 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:36.298091 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:36.298114 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:36.300509 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.301102 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.301105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.301107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.301197 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.301556 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.301558 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.301562 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.301792 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:36.302670 (+ 878us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:36.302749 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 11:14:36.302901 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4022}
Aug 12 11:14:36 dist-test-slave-drl0 krb5kdc[20653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786533275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.44.253@KRBTEST.COM
I20260812 11:14:36.304221 20915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 820895ecbc5941f2bf5349707ca681b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:14:36.305088 21040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.294587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.44.252:37947 (local address 127.0.0.1:45812)
0812 11:14:36.294916 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0812 11:14:36.294964 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 11:14:36.295036 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 11:14:36.295883 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 11:14:36.295889 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 11:14:36.296121 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0812 11:14:36.296815 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:36.296827 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.297772 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.297775 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:36.298486 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 11:14:36.298497 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.299437 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.299440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 11:14:36.299519 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:36.299984 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 11:14:36.299999 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 11:14:36.301459 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 11:14:36.303506 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:36.303508 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:36.303510 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:36.303743 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:36.304028 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 11:14:36.304030 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 11:14:36.304032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 11:14:36.304106 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 11:14:36.304519 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 11:14:36.304521 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 11:14:36.304605 (+ 84us) client_negotiation.cc:769] Sending connection context
0812 11:14:36.304705 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 11:14:36.304923 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":102,"threads_started":1}
I20260812 11:14:36.305670 21039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.295062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45812 (local address 127.17.44.252:37947)
0812 11:14:36.295350 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.295354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.295367 (+ 13us) server_negotiation.cc:407] Connection header received
0812 11:14:36.295437 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.295442 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.295625 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.295746 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.296954 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.297647 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.298603 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.299007 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.299103 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:36.301591 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:36.301595 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:36.301597 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:36.301619 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:36.303393 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.303850 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.303853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.303855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.303932 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.304189 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.304192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.304193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.304391 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:36.304462 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:36.305381 (+ 919us) server_negotiation.cc:299] Negotiation successful
0812 11:14:36.305511 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204}
I20260812 11:14:36.308542 20791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46: Bootstrap starting.
I20260812 11:14:36.309706 21033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 339d6aed629d406f9d4d4837fdcc2a02: Bootstrap starting.
I20260812 11:14:36.310503 20922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.294563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46264 (local address 127.17.44.253:45925)
0812 11:14:36.294709 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.294714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.295042 (+ 328us) server_negotiation.cc:407] Connection header received
0812 11:14:36.295283 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.295288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.295334 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.295421 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.296476 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.297233 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.298844 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.299211 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.299320 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:36.304124 (+ 4804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:36.304129 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:36.304131 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:36.304160 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:36.307283 (+ 3123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.308470 (+ 1187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.308475 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.308478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.308641 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.309148 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.309152 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.309154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.309410 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:36.309667 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:36.310192 (+ 525us) server_negotiation.cc:299] Negotiation successful
0812 11:14:36.310330 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 11:14:36.308321 20800 master_runner.cc:186] Error getting master registration for 127.17.44.254:33495: OK, instance_id { permanent_uuid: "af6c1021bab048c280b93b4d48c95b46" instance_seqno: 1786533275752932 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 11:14:36.312207 20800 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 11:14:36.312232 21033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 339d6aed629d406f9d4d4837fdcc2a02: Neither blocks nor log segments found. Creating new log.
I20260812 11:14:36.312239 20791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46: Neither blocks nor log segments found. Creating new log.
I20260812 11:14:36.312570 21044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 820895ecbc5941f2bf5349707ca681b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6c1021bab048c280b93b4d48c95b46" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33495 } } peers { permanent_uuid: "820895ecbc5941f2bf5349707ca681b6" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } } peers { permanent_uuid: "339d6aed629d406f9d4d4837fdcc2a02" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } } }
I20260812 11:14:36.312772 21044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 820895ecbc5941f2bf5349707ca681b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:14:36.313170 21033 log.cc:824] T 00000000000000000000000000000000 P 339d6aed629d406f9d4d4837fdcc2a02: Log is configured to *not* fsync() on all Append() calls
I20260812 11:14:36.313489 20791 log.cc:824] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46: Log is configured to *not* fsync() on all Append() calls
I20260812 11:14:36.314667 17587 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 11:14:36.315375 20791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46: No bootstrap required, opened a new log
I20260812 11:14:36.315375 21033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 339d6aed629d406f9d4d4837fdcc2a02: No bootstrap required, opened a new log
I20260812 11:14:36.316036 20741 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "af6c1021bab048c280b93b4d48c95b46"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43852
I20260812 11:14:36.319056 20791 raft_consensus.cc:348] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6c1021bab048c280b93b4d48c95b46" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33495 } } peers { permanent_uuid: "820895ecbc5941f2bf5349707ca681b6" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } } peers { permanent_uuid: "339d6aed629d406f9d4d4837fdcc2a02" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } }
I20260812 11:14:36.319303 20791 raft_consensus.cc:729] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af6c1021bab048c280b93b4d48c95b46, State: Initialized, Role: FOLLOWER
I20260812 11:14:36.319635 20791 consensus_queue.cc:260] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6c1021bab048c280b93b4d48c95b46" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33495 } } peers { permanent_uuid: "820895ecbc5941f2bf5349707ca681b6" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } } peers { permanent_uuid: "339d6aed629d406f9d4d4837fdcc2a02" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } }
W20260812 11:14:36.319801 21065 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 820895ecbc5941f2bf5349707ca681b6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:14:36.319872 21065 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:14:36.320218 21070 sys_catalog.cc:455] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6c1021bab048c280b93b4d48c95b46" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33495 } } peers { permanent_uuid: "820895ecbc5941f2bf5349707ca681b6" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } } peers { permanent_uuid: "339d6aed629d406f9d4d4837fdcc2a02" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } } }
I20260812 11:14:36.320501 21070 sys_catalog.cc:458] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:14:36.320976 20791 sys_catalog.cc:564] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:14:36.321066 20741 raft_consensus.cc:482] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 11:14:36.321174 20741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:14:36.321714 21033 raft_consensus.cc:348] T 00000000000000000000000000000000 P 339d6aed629d406f9d4d4837fdcc2a02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6c1021bab048c280b93b4d48c95b46" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33495 } } peers { permanent_uuid: "820895ecbc5941f2bf5349707ca681b6" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } } peers { permanent_uuid: "339d6aed629d406f9d4d4837fdcc2a02" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } }
I20260812 11:14:36.321972 21033 raft_consensus.cc:729] T 00000000000000000000000000000000 P 339d6aed629d406f9d4d4837fdcc2a02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 339d6aed629d406f9d4d4837fdcc2a02, State: Initialized, Role: FOLLOWER
I20260812 11:14:36.322440 20741 raft_consensus.cc:504] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6c1021bab048c280b93b4d48c95b46" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33495 } } peers { permanent_uuid: "820895ecbc5941f2bf5349707ca681b6" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } } peers { permanent_uuid: "339d6aed629d406f9d4d4837fdcc2a02" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } }
I20260812 11:14:36.322399 21033 consensus_queue.cc:260] T 00000000000000000000000000000000 P 339d6aed629d406f9d4d4837fdcc2a02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6c1021bab048c280b93b4d48c95b46" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33495 } } peers { permanent_uuid: "820895ecbc5941f2bf5349707ca681b6" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } } peers { permanent_uuid: "339d6aed629d406f9d4d4837fdcc2a02" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } }
I20260812 11:14:36.323433 20741 leader_election.cc:288] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46 [CANDIDATE]: Term 1 election: Requested vote from peers 820895ecbc5941f2bf5349707ca681b6 (127.17.44.253:45925), 339d6aed629d406f9d4d4837fdcc2a02 (127.17.44.252:37947)
I20260812 11:14:36.323803 21033 sys_catalog.cc:564] T 00000000000000000000000000000000 P 339d6aed629d406f9d4d4837fdcc2a02 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 11:14:36.324047 20996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "af6c1021bab048c280b93b4d48c95b46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "339d6aed629d406f9d4d4837fdcc2a02"
I20260812 11:14:36.324266 20996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 339d6aed629d406f9d4d4837fdcc2a02 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:14:36.324441 20873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "af6c1021bab048c280b93b4d48c95b46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "820895ecbc5941f2bf5349707ca681b6"
I20260812 11:14:36.324666 20873 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 820895ecbc5941f2bf5349707ca681b6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 11:14:36.325742 20996 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 339d6aed629d406f9d4d4837fdcc2a02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af6c1021bab048c280b93b4d48c95b46 in term 1.
I20260812 11:14:36.326094 20873 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 820895ecbc5941f2bf5349707ca681b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af6c1021bab048c280b93b4d48c95b46 in term 1.
I20260812 11:14:36.326723 20703 leader_election.cc:302] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 339d6aed629d406f9d4d4837fdcc2a02, af6c1021bab048c280b93b4d48c95b46; no voters:
W20260812 11:14:36.329185 21118 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 339d6aed629d406f9d4d4837fdcc2a02: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 11:14:36.329265 21118 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 11:14:36.331067 21070 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 11:14:36.331382 21070 raft_consensus.cc:686] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46 [term 1 LEADER]: Becoming Leader. State: Replica: af6c1021bab048c280b93b4d48c95b46, State: Running, Role: LEADER
I20260812 11:14:36.332464 21070 consensus_queue.cc:237] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6c1021bab048c280b93b4d48c95b46" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33495 } } peers { permanent_uuid: "820895ecbc5941f2bf5349707ca681b6" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } } peers { permanent_uuid: "339d6aed629d406f9d4d4837fdcc2a02" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } }
I20260812 11:14:36.333446 20924 master_runner.cc:263] No leader master found from master 339d6aed629d406f9d4d4837fdcc2a02
I20260812 11:14:36.333513 20924 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 11:14:36.334028 20676 master_runner.cc:263] No leader master found from master af6c1021bab048c280b93b4d48c95b46
I20260812 11:14:36.334096 20676 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 11:14:36.337926 21134 sys_catalog.cc:455] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af6c1021bab048c280b93b4d48c95b46. Latest consensus state: current_term: 1 leader_uuid: "af6c1021bab048c280b93b4d48c95b46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6c1021bab048c280b93b4d48c95b46" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33495 } } peers { permanent_uuid: "820895ecbc5941f2bf5349707ca681b6" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } } peers { permanent_uuid: "339d6aed629d406f9d4d4837fdcc2a02" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } } }
I20260812 11:14:36.338088 21134 sys_catalog.cc:458] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:36.340085 21139 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 11:14:36.342485 20922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.328352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46270 (local address 127.17.44.253:45925)
0812 11:14:36.328465 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.328470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.332111 (+ 3641us) server_negotiation.cc:407] Connection header received
0812 11:14:36.332332 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.332335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.332388 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.332446 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.334253 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.334889 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.336131 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.336494 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.336585 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:36.338518 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:36.338521 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:36.338524 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:36.338550 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:36.340346 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.340970 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.340980 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.340982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.341084 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.341391 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.341395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.341397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.341671 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:36.341839 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:36.342165 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 11:14:36.342308 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 11:14:36.343101 21080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 339d6aed629d406f9d4d4837fdcc2a02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6c1021bab048c280b93b4d48c95b46" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33495 } } peers { permanent_uuid: "820895ecbc5941f2bf5349707ca681b6" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } } peers { permanent_uuid: "339d6aed629d406f9d4d4837fdcc2a02" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } } }
I20260812 11:14:36.343253 21080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 339d6aed629d406f9d4d4837fdcc2a02 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:14:36.344818 21139 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 11:14:36.346786 20793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.330881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43866 (local address 127.17.44.254:33495)
0812 11:14:36.331008 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.331013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.337081 (+ 6068us) server_negotiation.cc:407] Connection header received
0812 11:14:36.337277 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.337281 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.337341 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.337430 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.338433 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.339253 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.340548 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.341016 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.341095 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:36.343650 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:36.343654 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:36.343656 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:36.343683 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:36.345144 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.345559 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.345563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.345565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.345649 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.345922 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.345927 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.345929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.346107 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:36.346250 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:36.346507 (+ 257us) server_negotiation.cc:299] Negotiation successful
0812 11:14:36.346625 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 11:14:36.351939 21041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:36.332130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45820 (local address 127.17.44.252:37947)
0812 11:14:36.332244 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.332248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.339193 (+ 6945us) server_negotiation.cc:407] Connection header received
0812 11:14:36.339375 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.339378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.339431 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.339484 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.340386 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.341091 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.342357 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:36.342660 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:36.342744 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:36.348565 (+ 5821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 11:14:36.348568 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 11:14:36.348570 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 11:14:36.348594 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 11:14:36.349972 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.350430 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.350433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.350435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.350519 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 11:14:36.350813 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 11:14:36.350817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 11:14:36.350819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 11:14:36.351079 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 11:14:36.351229 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:36.351625 (+ 396us) server_negotiation.cc:299] Negotiation successful
0812 11:14:36.351757 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 11:14:36.362128 20872 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 820895ecbc5941f2bf5349707ca681b6 [term 1 FOLLOWER]: Refusing update from remote peer af6c1021bab048c280b93b4d48c95b46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:14:36.362309 20996 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 339d6aed629d406f9d4d4837fdcc2a02 [term 1 FOLLOWER]: Refusing update from remote peer af6c1021bab048c280b93b4d48c95b46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 11:14:36.362821 21134 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "820895ecbc5941f2bf5349707ca681b6" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:14:36.363004 21070 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "339d6aed629d406f9d4d4837fdcc2a02" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 11:14:36.367653 21080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 339d6aed629d406f9d4d4837fdcc2a02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af6c1021bab048c280b93b4d48c95b46. Latest consensus state: current_term: 1 leader_uuid: "af6c1021bab048c280b93b4d48c95b46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6c1021bab048c280b93b4d48c95b46" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33495 } } peers { permanent_uuid: "820895ecbc5941f2bf5349707ca681b6" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } } peers { permanent_uuid: "339d6aed629d406f9d4d4837fdcc2a02" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } } }
I20260812 11:14:36.367795 21080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 339d6aed629d406f9d4d4837fdcc2a02 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:14:36.369211 21044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 820895ecbc5941f2bf5349707ca681b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af6c1021bab048c280b93b4d48c95b46. Latest consensus state: current_term: 1 leader_uuid: "af6c1021bab048c280b93b4d48c95b46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6c1021bab048c280b93b4d48c95b46" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33495 } } peers { permanent_uuid: "820895ecbc5941f2bf5349707ca681b6" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } } peers { permanent_uuid: "339d6aed629d406f9d4d4837fdcc2a02" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } } }
I20260812 11:14:36.369355 21044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 820895ecbc5941f2bf5349707ca681b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:14:36.373678 21134 sys_catalog.cc:455] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af6c1021bab048c280b93b4d48c95b46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6c1021bab048c280b93b4d48c95b46" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33495 } } peers { permanent_uuid: "820895ecbc5941f2bf5349707ca681b6" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } } peers { permanent_uuid: "339d6aed629d406f9d4d4837fdcc2a02" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } } }
I20260812 11:14:36.373819 21134 sys_catalog.cc:458] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:36.374994 21044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 820895ecbc5941f2bf5349707ca681b6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af6c1021bab048c280b93b4d48c95b46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6c1021bab048c280b93b4d48c95b46" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33495 } } peers { permanent_uuid: "820895ecbc5941f2bf5349707ca681b6" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } } peers { permanent_uuid: "339d6aed629d406f9d4d4837fdcc2a02" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } } }
I20260812 11:14:36.375133 21044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 820895ecbc5941f2bf5349707ca681b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:14:36.375341 21141 mvcc.cc:205] Tried to move back new op lower bound from 7317640299971911680 to 7317640299863322624. Current Snapshot: MvccSnapshot[applied={T|T < 7317640299971911680}]
I20260812 11:14:36.380983 21070 sys_catalog.cc:455] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af6c1021bab048c280b93b4d48c95b46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6c1021bab048c280b93b4d48c95b46" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33495 } } peers { permanent_uuid: "820895ecbc5941f2bf5349707ca681b6" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } } peers { permanent_uuid: "339d6aed629d406f9d4d4837fdcc2a02" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } } }
I20260812 11:14:36.381112 21070 sys_catalog.cc:458] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46 [sys.catalog]: This master's current role is: LEADER
I20260812 11:14:36.382081 21080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 339d6aed629d406f9d4d4837fdcc2a02 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af6c1021bab048c280b93b4d48c95b46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af6c1021bab048c280b93b4d48c95b46" member_type: VOTER last_known_addr { host: "127.17.44.254" port: 33495 } } peers { permanent_uuid: "820895ecbc5941f2bf5349707ca681b6" member_type: VOTER last_known_addr { host: "127.17.44.253" port: 45925 } } peers { permanent_uuid: "339d6aed629d406f9d4d4837fdcc2a02" member_type: VOTER last_known_addr { host: "127.17.44.252" port: 37947 } } }
I20260812 11:14:36.382211 21080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 339d6aed629d406f9d4d4837fdcc2a02 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 11:14:36.382512 21142 mvcc.cc:205] Tried to move back new op lower bound from 7317640299971911680 to 7317640299863322624. Current Snapshot: MvccSnapshot[applied={T|T < 7317640299971911680}]
I20260812 11:14:36.383824 21139 catalog_manager.cc:1347] Generated new cluster ID: 5a948b5898b94bacb992379a180a016a
I20260812 11:14:36.383899 21139 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 11:14:36.407846 21139 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 11:14:36.409641 21139 catalog_manager.cc:1504] Loading token signing keys...
I20260812 11:14:36.421542 21139 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af6c1021bab048c280b93b4d48c95b46: Generated new TSK 0
I20260812 11:14:36.422580 21139 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 11:14:36.464097 20922 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:36.460905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46280 (local address 127.17.44.253:45925)
0812 11:14:36.461057 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.461062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.463000 (+ 1938us) server_negotiation.cc:407] Connection header received
0812 11:14:36.463200 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.463203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.463253 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.463305 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.463952 (+ 647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46280: BlockingRecv error: recv got EOF from 127.0.0.1:46280 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260812 11:14:36.465456 20793 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:36.461785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43868 (local address 127.17.44.254:33495)
0812 11:14:36.461898 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.461902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.464573 (+ 2671us) server_negotiation.cc:407] Connection header received
0812 11:14:36.464740 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.464743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.464790 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.464840 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.465327 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43868: BlockingRecv error: recv got EOF from 127.0.0.1:43868 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260812 11:14:36.466747 21041 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:36.462404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45822 (local address 127.17.44.252:37947)
0812 11:14:36.462518 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.462522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.465893 (+ 3371us) server_negotiation.cc:407] Connection header received
0812 11:14:36.466061 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.466064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.466109 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.466158 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.466634 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45822: BlockingRecv error: recv got EOF from 127.0.0.1:45822 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 11:14:36.471282 20922 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:36.469717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46282 (local address 127.17.44.253:45925)
0812 11:14:36.469813 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.469817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.470318 (+ 501us) server_negotiation.cc:407] Connection header received
0812 11:14:36.470545 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.470548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.470621 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.470670 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.471187 (+ 517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46282: BlockingRecv error: recv got EOF from 127.0.0.1:46282 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260812 11:14:36.476548 20922 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:36.475183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46290 (local address 127.17.44.253:45925)
0812 11:14:36.475301 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.475305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.475705 (+ 400us) server_negotiation.cc:407] Connection header received
0812 11:14:36.475880 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.475883 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.475928 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.475976 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.476460 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46290: BlockingRecv error: recv got EOF from 127.0.0.1:46290 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260812 11:14:36.481823 20922 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:36.480472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46304 (local address 127.17.44.253:45925)
0812 11:14:36.480562 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.480566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.481026 (+ 460us) server_negotiation.cc:407] Connection header received
0812 11:14:36.481195 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.481199 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.481242 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.481288 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.481734 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46304: BlockingRecv error: recv got EOF from 127.0.0.1:46304 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260812 11:14:36.492251 20922 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:36.487880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46318 (local address 127.17.44.253:45925)
0812 11:14:36.488013 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.488017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.491344 (+ 3327us) server_negotiation.cc:407] Connection header received
0812 11:14:36.491546 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.491549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.491597 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.491646 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.492154 (+ 508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46318: BlockingRecv error: recv got EOF from 127.0.0.1:46318 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260812 11:14:36.504323 20922 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:36.502641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46326 (local address 127.17.44.253:45925)
0812 11:14:36.502757 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.502762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.503373 (+ 611us) server_negotiation.cc:407] Connection header received
0812 11:14:36.503589 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.503592 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.503644 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.503695 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.504217 (+ 522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46326: BlockingRecv error: recv got EOF from 127.0.0.1:46326 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260812 11:14:36.524672 20922 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:36.522891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46334 (local address 127.17.44.253:45925)
0812 11:14:36.523033 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.523040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.523591 (+ 551us) server_negotiation.cc:407] Connection header received
0812 11:14:36.523796 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.523801 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.523856 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.523912 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.524521 (+ 609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46334: BlockingRecv error: recv got EOF from 127.0.0.1:46334 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260812 11:14:36.561190 20922 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:36.559494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46342 (local address 127.17.44.253:45925)
0812 11:14:36.559619 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.559623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.560165 (+ 542us) server_negotiation.cc:407] Connection header received
0812 11:14:36.560362 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.560365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.560421 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.560473 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.561070 (+ 597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46342: BlockingRecv error: recv got EOF from 127.0.0.1:46342 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 11:14:36.630452 20922 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:36.628706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46356 (local address 127.17.44.253:45925)
0812 11:14:36.628891 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.628895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.629278 (+ 383us) server_negotiation.cc:407] Connection header received
0812 11:14:36.629396 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.629399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.629464 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.629563 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.630337 (+ 774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46356: BlockingRecv error: recv got EOF from 127.0.0.1:46356 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260812 11:14:36.763054 20922 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:36.761281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46368 (local address 127.17.44.253:45925)
0812 11:14:36.761473 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:36.761481 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:36.761755 (+ 274us) server_negotiation.cc:407] Connection header received
0812 11:14:36.761865 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:36.761868 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:36.761930 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:36.762015 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:36.762828 (+ 813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46368: BlockingRecv error: recv got EOF from 127.0.0.1:46368 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260812 11:14:37.023306 20922 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 11:14:37.021976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46380 (local address 127.17.44.253:45925)
0812 11:14:37.022152 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:37.022159 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:37.022355 (+ 196us) server_negotiation.cc:407] Connection header received
0812 11:14:37.022437 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:37.022439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:37.022492 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:37.022579 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 11:14:37.023188 (+ 609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46380: BlockingRecv error: recv got EOF from 127.0.0.1:46380 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 11:14:37.321844 21065 catalog_manager.cc:1259] Loaded cluster ID: 5a948b5898b94bacb992379a180a016a
I20260812 11:14:37.321978 21065 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 820895ecbc5941f2bf5349707ca681b6: loading cluster ID for follower catalog manager: success
I20260812 11:14:37.324445 21065 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 820895ecbc5941f2bf5349707ca681b6: acquiring CA information for follower catalog manager: success
I20260812 11:14:37.325855 21065 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 820895ecbc5941f2bf5349707ca681b6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 11:14:37.330368 21118 catalog_manager.cc:1259] Loaded cluster ID: 5a948b5898b94bacb992379a180a016a
I20260812 11:14:37.330470 21118 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 339d6aed629d406f9d4d4837fdcc2a02: loading cluster ID for follower catalog manager: success
I20260812 11:14:37.332186 21118 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 339d6aed629d406f9d4d4837fdcc2a02: acquiring CA information for follower catalog manager: success
I20260812 11:14:37.333216 21118 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 339d6aed629d406f9d4d4837fdcc2a02: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 11:14:37.542706 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:37.538334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46390 (local address 127.17.44.253:45925)
0812 11:14:37.538595 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:37.538599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:37.538636 (+ 37us) server_negotiation.cc:407] Connection header received
0812 11:14:37.538694 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:37.538696 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:37.538828 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:37.538921 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:14:37.539290 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:37.539852 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:37.540805 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:37.541134 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:37.541276 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:37.541641 (+ 365us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:14:37.542119 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:14:37.542327 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:37.542367 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 11:14:37.542525 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":110,"threads_started":1}
I20260812 11:14:37.550135 21266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:37.546522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45824 (local address 127.17.44.252:37947)
0812 11:14:37.546784 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:37.546787 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:37.546799 (+ 12us) server_negotiation.cc:407] Connection header received
0812 11:14:37.546876 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:37.546885 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:37.547018 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:37.547151 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:14:37.547366 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:37.547865 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:37.548697 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:37.548928 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:37.549043 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:37.549335 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:14:37.549722 (+ 387us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:14:37.549856 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:37.549925 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 11:14:37.549976 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":74,"threads_started":1}
I20260812 11:14:37.557730 21266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:37.553736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45826 (local address 127.17.44.252:37947)
0812 11:14:37.553854 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:37.553861 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:37.554228 (+ 367us) server_negotiation.cc:407] Connection header received
0812 11:14:37.554377 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:37.554380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:37.554446 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:37.554495 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:14:37.554866 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:37.555630 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:37.556561 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:37.556858 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:37.556959 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:37.557213 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:14:37.557286 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:14:37.557353 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:37.557527 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 11:14:37.557578 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 11:14:37.557993 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:37.553414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46404 (local address 127.17.44.253:45925)
0812 11:14:37.553548 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:37.553561 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:37.554039 (+ 478us) server_negotiation.cc:407] Connection header received
0812 11:14:37.554232 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:37.554236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:37.554307 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:37.554379 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:14:37.554690 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:37.555536 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:37.556716 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:37.557044 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:37.557151 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:37.557489 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:14:37.557573 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:14:37.557640 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:37.557793 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 11:14:37.557851 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 11:14:37.558125 21278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 11:14:37.553366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43884 (local address 127.17.44.254:33495)
0812 11:14:37.553763 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0812 11:14:37.553768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 11:14:37.553791 (+ 23us) server_negotiation.cc:407] Connection header received
0812 11:14:37.553989 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 11:14:37.553995 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 11:14:37.554179 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 11:14:37.554304 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 11:14:37.554526 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:37.555276 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:37.556411 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 11:14:37.556748 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 11:14:37.556854 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 11:14:37.557179 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 11:14:37.557507 (+ 328us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 11:14:37.557862 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0812 11:14:37.557902 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 11:14:37.557980 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":99,"threads_started":1}
I20260812 11:14:37.566000 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 20676
I20260812 11:14:37.581578 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 20800
I20260812 11:14:37.595034 17587 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqOACJ0/build/debug/bin/kudu with pid 20924
2026-08-12T11:14:37Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3495 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6978 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (315864 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST